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DECISION  MAKING:  AN  ANNOTATED  BIBLIOGRAPHY  OF  SELECTED  LITERATURE 

EXECUTIVE  SUMMARY 


Requirement: 

To  provide  a  guide  for  research  and  military  personnel  interested  in 
current  literature  on  decision  making. 


Procedure: 

The  literature  collected  was  selective  in  nature,  but  has  wide  applicabil¬ 
ity  in  the  area  of  decision  making  pertaining  to  basic  research,  decision  aids, 
evaluation,  and  literature  reviews/bibliographies. 

The  annotated  bibliography  consists  of  papers  chosen  from  technical  re¬ 
ports,  published  scientific  studies,  presented  papers,  and  unpublished  reports 
in  progress.  A  categorical  index  is  included. 


Findings : 

The  literature  is  presented  in  the  following  manner: 

a.  According  to  categories  as  specified  in  the  Table  of  Contents. 

b.  Indexed  alphabetically  by  first  author  within  each  category. 

c.  Indexed  alphabetically  by  author  within  year  of  publication. 

The  classification  scheme  is  representative  of  the  contents  of  the  decision 
making  material.  The  abstracts  are  designed  to  allow  the  reader  an  objective 
preview  of  each  article.  When  a  reference  applies  to  more  than  one  category, 
it  is  repeated. 


Utilization  of  Findings: 

This  bibliography  can  be  used  by  system  analysts,  system  developers,  and 
researchers  doing  work  in  decision  making  as  an  aid  in  identifying  relevant 
issues  and  in  locating  related  research  findings.  As  such,  it  will  facilitate 
the  conduct  of  research,  building  on  the  past  work  of  others,  and  contributing 
to  the  efficient  use  of  resources. 
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DECISION  MAKING: 


AN  ANNOTATED  BIBLIOGRAPHY  OF  SELECTED  LITERATURE 


INTRODUCTION 


Decision  aiding  was  identified  by  the  1983  Army  Command  and  Control  Master 
Plan  as  an  area  requiring  increased  attention.  Interest  in  decision  aiding  and 
in  the  development  of  tactical  decision  aids  has  increased  significantly  over 
the  past  several  years,  and  decision  aiding  is  now  a  major  concern  in  the  re¬ 
search  and  development  community.  This  research  is  critical  to  the  design, 
a/aluation,  and  implementation  of  decision  support  systems.  Those  from  the 
many  disciplines  working  in  the  area  of  decision  making  need  to  make  efficient 
use  of  past  research  findings.  Using  this  bibliography  researchers  can  survey 
potentially  relevant  topics  and  identify  articles  for  more  detailed  examina¬ 
tion. 

The  body  of  literature  relating  to  decision  making  is  enormous,  totaling 
in  the  thousands  of  articles.  This  bibliography  contains  articles  that  have 
been  collected  in  the  course  of  the  ARI  Fort  Leavenworth  Field  Unit's  continu¬ 
ing  research  program  in  decision  aiding.  Literature  selection  was  neither 
comprehensive  nor  completely  representative,  however,  it  did  include  a  wide 
variety  of  articles  relating  to  the  most  important  issues  facing  decision  aid 
designers  and  evaluators. 

The  report  has  been  divided  into  four  main  categories:  basic  research, 
decision  aids,  evaluation,  and  reviews  and  bibliographies.  Subheadings  are 
identified  under  each  category  except  the  fourth.  In  addition  to  the  annotated 
entries,  a  topic  and  author  index  is  included  at  the  back  of  this  report. 
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DECISION  MAKING:  AN  ANNOTATED  BIBLIOGRAPHY  OF  SELECTED  LITERATURE 


BASIC  RESEARCH 


General  Decision  Making 


Alter,  S.  (1977,  February).  Why  is  man-computer  interaction  important  for 
decision  support  systems?  Interfaces,  7/2),  109-115* 

Argues  that  interactive,  conversational  decision  support  systems  do  not 
represent  their  true  identities  in  many  cases.  The  author  believes  that  better 
man-computer  interfaces  are  needed,  but  not  necessarily  by  managers  themselves. 
Suggests  that  the  focus  should  be  on  experts  who  understand  the  details  under¬ 
lying  a  support  system;  if  managers  do  not  learn  the  correct  usage  of  decision 
support  systems,  then  usage  should  be  discouraged  without  the  aid  of  an  inter¬ 
mediary. 

Arkes,  H.R. ,  Harkness,  A.R.  (1980).  Effect  of  making  a  diagnosis  on  subsequent 
recognition  of  symptoms.  Journal  of  Experimental  Psychology:  Human  Learning 
and  Memory,  6/5),  568-575* 

A  "false  recognition  effect"  is  described,  whereby  a  subject  incorrectly 
"remembers"  items  which  remain  consistent  with  his  "schema"  of  something.  The 
authors  concur  with  previous  research  results  that  a  diagnosis  functions  as  a 
schema.  Three  experiments  were  conducted  to  observe  the  effect  of  the  posses¬ 
sion  of  a  diagnosis  on  consequent  recognition  of  symptoms  that  were  conceptu¬ 
ally  related/unrelated  to  the  diagnosis.  Memory  distortion  may  be  attributed 
to  overconfidence  in  the  validity  of  one's  hypothesis,  false  recognition  of 
unpresented  inferences  or  features  of  a  schema,  or  a  long  retention  interval. 

Bandyopadhyay ,  R.  (1977,  January).  Information  for  organizational  decision¬ 
making  -  a  literature  review.  IEEE  Transactions  on  Systems,  Man,  and 
Cybernetics,  SMC-7(l ) ,  1-15* 

Classifies  literature  pertaining  to  organizational  decision  making  into 
three  areas:  management  science,  classical,  and  behavioral  science.  Stresses 
that  there  is  a  problem  in  measuring  information  value.  The  author  aims  to 
provide  insight  into  the  broad  area  cf  organizational  decisionmaking. 

Beach,  L.R.  (1966,  March).  Accuracy  and  consistency  in  the  revision  of 
subjective  probabilities.  IEEE  Transactions  on  Human  Factors  in  Electronics,  J_ 
(0  ,  29-36. 

This  report  investigates  the  consistency  with  which  subjective  probabili¬ 
ties  are  revised.  Four  experiments  were  performed  in  which  models  of  probabil¬ 
istic  concept  formation  studies  and  Bayesian  subjective  probability  revision 
studies  were  combined.  Results  showed  that  subjects'  revisions  of  their  sub¬ 
jective  probabilities  can  be  accurately  predicted  with  Bayes'  theorem. 
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Bettenhausen,  K. ,  &  Murnighan,  J.K.  (1985).  The  emergence  of  norms  in  com¬ 
petitive  decisionmaking  groups.  Administrative  Science  Quarterly,  30,  350-372. 

Interaction  among  group  members  in  nineteen  newly  formed  decision  making 
groups  (composed  of  five  persons  each)  was  studied  to  understand  how  social 
norms  form.  The  behavior  of  these  group  members  served  as  the  foundation  in 
constructing  a  model  of  norm  development  whereby  uncertainty  over  appropriate 
behavior  directs  people  to  utilize  their  past  experiences  in  similar  social 
settings  as  scripts.  By  using  these  experiences  they  determine  behaviors  in 
their  current  situations.  The  results  of  gaining  knowledge  about  how  norms 
develop  would  grant  the  capability  of  "managing"  this  limitation  in  group  deci¬ 
sion  making  situations. 

Blood,  M.R.  (1971).  Short  notes  -  the  validity  of  importance.  Journal  of 
Applied  Psychology,  55(5),  487-488. 

Debates  the  utility  of  weighting  by  importance  in  measuring  job  satisfac¬ 
tion  when  combining  measures  of  satisfaction  relevant  to  job  aspects  into  a 
measure  of  aggregate  satisfaction.  Doubts  the  validity  of  information  which  is 
applied  in  importance  weighting. 

Brown,  R.V.  (1974,  October).  Modeling  subsequent  acts  for  decision  analysis: 
a  heretical  view.  Paper  presented  at  International  Conference  of  ORSA/TIMS, 
Puerto  Rico. 

Addresses  problems  in  the  technology  of  decision  analysis,  namely  structur¬ 
ing  of  the  model  itself.  The  conventional  decision  theory  model  in  the  form  of 
a  decision  tree  is  referred  to  as  "preposterior"  analysis,  which  is  faulted  by 
the  author  as  rarely  meeting  "real  world"  decision  analyses  requirements. 

This  could  possibly  be  a  reason  for  the  lack  of  success  with  decision  analysis 
in  information  decisions  as  opposed  to  successful  operating  decisions  which  are 
all  based  on  prior  information.  An  acts-as-events  model  is  proposed  which 
supposedly  provides  an  adequate  framework  for  assessment  (and  involves  a 
smaller  number  of  total  assessments)  and  allows  events  to  be  more  readily  visu¬ 
alized. 


Chu,  Y. ,  &  Rouse,  W.B.  (1979,  December).  Adaptive  allocation  of  decisionmaking 
responsibility  between  human  and  computer  in  multitask  situations.  IEEE 
Transactions  on  Systems,  Man,  and  Cybernetics,  SMC-9(12),  769-778. 

Addresses  the  question  of  how  decisionmaking  responsibilities  should  be 
allocated  between  humans  and  increasingly  intelligent  computers  as  systems 
become  more  automated.  The  authors  maintain  that  with  "adaptive  allocation" 
the  allocation  of  responsibility  is  contingent  upon  the  state  of  the  system  and 
decisionmakers.  The  decisionmaker  most  capable  of  performing  a  task  at  a  par¬ 
ticular  time  will  be  subject  to  complete  it.  An  experimental  study  involving  a 
flight  management  task  illustrates  this  general  strategy. 


Connelly,  D.P. ,  &  Johnson,  P.E.  (1980,  September), 
process.  Human  Pathology,  11(5),  412-419. 

Addresses  the  medical  problem  solving  process, 
making  phases  which  characterize  this  process  are: 
rect  diagnosis  to  a  level  adequate  for  therapeutic 


The  medical  problem  solving 

The  two  major  decision 
specification  of  the  cor- 
considerations,  and  choice 
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of  a  therapy  that  positively  influences  the  patient's  problem.  For  effective 
decision  making,  appropriate  selectivity  must  be  engaged  in  to  limit  the  amount 
of  information  worthy  of  examination  or  generated  from  the  thought  process. 

Connolly,  T. ,  &  Miklausich,  V.M.  (1978).  Some  effects  of  feedback  error  in 
diagnostic  decision  tasks.  Academy  of  Management  Journal,  21 (2) ,  301-307. 

A  laboratory  study  was  conducted  to  examine  the  effects  of  varying  the 
levels  of  feedback  error  on  performance  in  diagnostic  tasks.  Results  suggest 
that  the  comprehensive  decision  quality  is  contingent  on  both  the  quality  of 
information  available  when  the  decision  is  made  and  the  quality  of  subsequent 
feedback. 

Cyert,  R.M. ,  &  DeGroot,  M.H.  (1974,  May/June).  Rational  expectations  and 
Bayesian  analysis.  Journal  of  Political  Economy,  82,  521-536. 

Models  that  delineate  the  means  by  which  rational  expectations  may  be  de¬ 
veloped  within  a  market  are  the  focus  of  this  article.  Three  distinct  models 
are  unfolded  which  explain  the  interaction  of  Bayesian  learning  and  the  expec¬ 
tations  in  achieving  a  market  equilibrium.  These  dynamic  models  depict  the 
transition  process  toward  equilibrium. 

Dino,  G.A. ,  &  Shanteau,  J.  (1984,  November).  What  skills  do  managers  consider 
important  for  effective  decision  making?  Paper  presented  at  the  Psychonomics 
Meeting,  San  Antonio,  Texas. 

This  study  was  conducted  with  the  purpose  of  analyzing  how  decision  makers 
themselves  define  decision  making  competence.  Thirty- five  bottom-level  and 
twenty-four  top-level  managers  participated  in  the  following:  (l)  Rated  skills 
that  they  believed  important  for  effective  decision  makers;  (2)  Rated  the 
importance  of  these  skills  in  upper- level  and  lower-level  decision  makers;  and, 
(3)  Were  evaluated  in  regards  to  the  effects  of  gender,  their  types  of 
organizations,  and  their  levels  within  their  organizations. 

Edwards,  W.  (1975,  August).  Research  on  the  technology  of  inference  and  deci¬ 
sion  (Report  No.  001597-F).  Los  Angeles,  California:  Advanced  Research 
Projects  Agency. 

The  technology  of  inference  and  decision  is  the  topic  of  this  research  re¬ 
port.  The  focus  is  on  the  elicitation  of  subjective  probabilities,  multi-at¬ 
tribute  utility  theory,  and  error  in  decision  analysis.  Stressed  is  the 
necessity  of  prioritizing  the  structuring  of  the  problem  along  with  processing 
information  and  the  elicitation  of  probabilities/utilities,  respectively. 

Edwards,  W.  (1977,  May).  How  to  use  multiattribute  utility  measurement  for 
social  decisionmaking.  IEEE  Transactions  on  Systems,  Man,  and  Cybernetics, 
SMC-7(5),  326-339.  .  "  "  ~ . . 

Contends  that  arguments  over  public  policy  often  result  from  dissension 
over  values.  These  disagreements  are  repeatedly  contested  in  the  context  of 
specific  decisions,  at  tremendous  social  cost  for  each  decision  which  must  be 
made.  The  author  utilizes  Multivariate  Utility  Measurement  (MAUM)  for  social 
decision  making  through  three  examples  in  applied  settings,  in  which  he  illus¬ 
trates  this  ten-step  process  to  elucidate  the  values  of  each  participant  in  the 
decision  making  process.  The  fundamental  idea  is  that  every  outcome  of  an 
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action  may  have  value  on  a  number  of  various  dimensions.  It  is  strongly  empha¬ 
sized  that  simplicity  in  elicitation  procedures  enable  social  policies  to  be 
efficiently  and  more  easily  implemented.  MAUM  is  also  flexible  in  response  to 
ever-changing  value  systems. 

Einhorn,  H.J.  &  Hogarth,  R.M.  (1981 ).  Behavioral  decision  theory:  processes 
of  judgment  and  choice.  Ann.  Rev.  Psychol.,  32,  53-88. 

This  review  of  behavioral  decision  theory  literature  argues  that  the 
judgmental  biases  exhibited  in  laboratory  situations  may  be  functional  in  natu¬ 
ral  settings.  Areas  reviewed  include  attention,  memory,  learning,  feedback, 
cognitive  representation,  and  conflict.  Assuming  a  broad  perspective  in  the 
research  of  decision  making  may  yield  the  most  insightful  and  generalized  re¬ 
sults. 

Einhorn,  H.J. ,  Hogarth,  R.M.,  &  Klempner,  E.  (1977).  Quality  of  group  judg¬ 
ment.  Psychological  Bulletin,  84,  158-172. 

Deals  with  the  accuracy  of  group  judgment  within  and  between  different 
populations  of  groups.  The  quality  of  group  judgment  is  defined.  Results  as 
well  as  constraints  affecting  judgmental  quality  of  various  strategies  for 
merging  opinions  under  a  diversity  of  circumstances  are  discussed.  A  statisti¬ 
cal  procedure  is  introduced  that  considers  which  baselines  are  suitable  in  the 
evaluation  of  the  quality  of  group  judgment. 

Farquhar,  P.H.  (1982).  Utility  assessment  methods  (Working  Paper  81-5). 

Davis,  California:  University  of  California. 

Attempts  to  integrate  current  methods  for  evaluating  single-attribute  util¬ 
ity  functions.  Presents  various  new  methods  of  evaluation  which  may  be  benefi¬ 
cial  in  furthering  the  research  in  this  area,  emphasizing  the  varied  steps  in 
the  utility  assessment  process,  and  contrasting  methods  of  comparative  risk. 
Reviews  approximately  twenty- four  utility  assessment  methods. 

Fischer,  G.W.  (1979,  July).  Utility  models  for  multiple  objective  decisions: 
do  they  accurately  represent  human  preferences?  The  Journal  for  the  American 
Institute  for  Decision  Sciences,  10(3) ,  451-479* 

This  paper  is  written  for  those  decision  scientists  who  do  not  utilize 
Multivariate  Utility  (MAU)  modeling  procedures  because  they  doubt  that  these 
procedures  are  reliable  in  providing  a  true  representation  of  human  prefer¬ 
ences.  The  author's  goal  is  to  persuade  many  that  the  MAU  approach  has  many 
useful  applications.  Two  approaches  to  MAU  modeling  as  well  as  relevant  psy¬ 
chological  research  are  examined.  Covered  in  this  article  are  the  following 
topics:  assigning  utility  values  to  decision  outcomes,  holistic  utility  assess¬ 
ment  methods,  an  overview  of  MAU  theory,  methods  for  assessing  decomposed  MAU 
functions,  common  objections  to  MAU  models,  and  validating  MAU  models  and  as¬ 
sessment  procedures. 
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Fischhoff,  B.  (1977,  May).  Decision  analysis:  clinical  art  or  clinical  sci¬ 
ence?  (Technical  Report  PTR- 1042-77-5).  Eugene,  Oregon:  Decision  Research,  A 
Branch  of  Perceptronics. 

Draws  an  analogy  between  decision  analysis  and  psychotherapy.  Areas  dis¬ 
cussed  include  evaluating  the  validity,  effectiveness,  and  underlying  assump¬ 
tions  of  decision  analysis,  improving  analysts'  skills,  and  social,  ethical, 
and  political  issues.  The  comparison  with  psychotherapy  serves  to  delineate 
the  important  issues  that  should  be  addressed  in  a  research  program. 

Fischhoff,  B.  (1985,  Winter).  Environmental  reporting:  what  to  ask  the 
experts.  The  Journalist,  11-15. 

This  study  of  environmental  reporting  focuses  on  the  difficulty  in  obtain¬ 
ing  complete,  relevant,  and  valid  environmental  stories.  The  author  cites  the 
major  problem  as  one  of  overconfidence  displayed  by  specialists  who  do  not 
realize  the  limits  to  their  own  understanding. 

Fischhoff,  B. ,  Slovic,  P. ,  &  Lichtenstein,  S.  (1978).  Fault  trees:  sensitiv¬ 
ity  of  estimated  failure  probabilities  to  problem  representation.  Journal  of 
Experimental  Psychology:  Human  Perception  and  Performance,  4^2),  530-344. 

This  study  analyzes  the  effects  of  varying  three  aspects  of  fault  tree 
structure  on  the  assessment  of  a  fault  tree  for  the  event  of  "a  car  failing  to 
start."  Results  confirmed  that:  (a)  People  were  insensitive  to  what  had  been 
omitted  from  the  fault  tree;  (b)  When  the  amount  of  detail  was  increased  this 
did  not  significantly  affect  perceptions;  and,  (c)  The  perceived  importance  of 
a  certain  branch  was  increased  by  presenting  it  as  two  separate  component 
branches. 

Gaeth,  G.J. ,  &  Shanteau,  J.  (1984).  Reducing  the  influence  of  irrelevant  in¬ 
formation  on  experienced  decision  makers.  Organizational  Behavior  and  Human 
Performance,  33,  263-282. 

Two  training  procedures  were  applied  to  assess  the  impact  on  reducing  the 
influence  of  irrelevant  information  on  agricultural  expert  judges.  The 
training  consisted  of  either  a  lecture  or  an  involved  interaction  session  with 
practice.  The  interactive  training  significantly  lessened  the  incorporation  of 
irrelevant  information  into  the  decision  made  by  the  expert  decision  makers. 

Not  only  did  it  also  improve  the  accuracy  of  the  judgments,  it  had  long  term 
effects.  Application  to  areas  other  than  those  which  are  perceptually  based  is 
advocated. 

Goitein,  B.  (1984).  The  danger  of  disappearing  postdecision  surprise:  comment 
on  Harrison  and  March,  "decision  making  and  postdecision  surprises."  Adminis¬ 
trative  Science  Quarterly,  29,  410-413. 

Comments  on  paper  written  by  Harrison  and  March  on  "Decisionmaking  and 
Postdecision  surprises"  (1984).  The  author  examines  the  issue  and  explains  why 
there  is  a  problem  with  the  disappearance  of  post  decision  surprise.  Believes 
that  "hindsight  bias,"  where  an  outcome  of  a  decision  may  be  seen  by  one  in 
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hindsight  as  having  been  expected  to  a  greater  extent  than  it  actually  was,  is 
detrimental  because  it  prevents  individual  and  organizational  learning  about 
faults  in  the  decision  system.  Further  research  is  needed  on  how  to  reduce 
hindsight  bias. 

Goldstein,  I.P.  (1979)-  The  genetic  graph:  a  representation  for  the  evolution 
of  procedural  knowledge.  International  Journal  of  Man-Machine  Studies,  1 1  , 
51-77.  ~~~ . . .  . . . 

A  model  of  the  evolution  of  rule-structured  knowledge  which  is  considered  a 
cornerstone  of  the  development  of  computer-based  coaches  is  illustrated.  A 
graph  structure  depicting  the  concepts  of  generalization,  correction,  and  re¬ 
finement  is  displayed  and  defined.  Various  genetic  graph  formulations  pertain¬ 
ing  to  reasoning  skills  used  in  playing  an  elementary  mathematical  game  are 
examined. 

Hart,  S.L.  (1985).  Toward  quality  criteria  for  collective  judgments. 
Organizational  Behavior  and  Human  Decision  Processes,  36,  209-228. 

Addresses  the  subject  of  group  decision  making  by  presenting  evaluative 
criteria,  which  were  derived  from  a  review  of  prevailing  literature,  for  judg¬ 
ing  when  a  "high-quality  outcome"  has  been  reached.  A  major  goal  which  is 
hoped  to  be  generated  by  this  project  is  the  construction  of  a  set  of  empiri¬ 
cally  derived  criteria  against  which  outcome  quality  may  be  judged.  The  most 
effective  procedures  for  the  achievement  of  a  high  quality  collective  judgment 
are  suggested  as  a  result  of  six  factual  group  problem-solving  cases  which 
were  evaluated  by  use  of  a  specially  developed  questionnaire. 

Hogarth,  R.M.  (1975,  September).  Methods  for  aggregated  opinions.  Revised 
version  of  paper  presented  at  the  Fifth  Research  Conference  on  Subjective 
Probability,  Utility,  and  Decision  Making,  Darmstadt,  West  Germany. 

Contends  that  there  is  no  "one  best  way"  to  aggregate  opinions.  Items 
discussed  in  this  report  include:  (a)  methods  and  models  for  aggregating  opin¬ 
ions  in  the  way  of  point  estimates  and  probability  distributions;  and  (b) 
causes  of  incompetency  in  interacting  groups  and  how  to  enhance  group 
processes. 

Hogarth,  R.M.  (1982,  March).  On  the  surprise  and  delight  of  inconsistent  re¬ 
sponses.  In  R.  Hogarth  (Ed.),  New  Directions  for  Methodology  of  Social  and 
Behavioral  Science:  Questions  Framing  and  Response  Consistency  (pp.  91-105). 
San  Francisco:  Josey  Bass. 

The  author  begins  by  providing  the  reader  with  various  definitions  of  the 
word  inconsistency,  and  relates  inconsistency  to  the  concept  of  variability. 

The  goals  and  steps  of  practical  inference  are  discussed,  and  sources  of  con¬ 
sistency  and  inconsistency  in  the  process  of  formulating  hypotheses  are  cited. 

Hogarth,  R.M.,  &  Makridakis,  S.  (1981 ).  The  value  of  decision  making  in  a 
complex  environment:  an  experimental  approach.  Management  Science,  27 , 

93-107.  . . 

An  experiment  was  conducted  in  which  a  competitive  game  was  played  to  as¬ 
sess  whether  or  not  the  costs  of  time  and  effort  used  to  analyze  a  decision 
outweighed  the  benefits.  Another  objective  was  to  discover  strategies  to 
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improve  decision  performance.  There  were  two  kinds  of  uncomplicated  decision 
rules:  (l)  rules  were  applied  consistently  (arbitrary-consistent),  and  (2) 
rules  were  subject  to  a  random  component  ( arbitrary- rand om ) . 

Howard,  R.A.  (1979,  October).  An  assessment  of  decision  analysis.  Stanford, 
California:  Stanford  University. 

Presents  an  assessment  of  decision  analysis  by  discussing  human  decision 
making,  decision  analysis,  the  usefulness  of  decision  analysis,  ethical  con¬ 
cerns,  and  the  challenge  that  must  be  faced  if  the  field  is  to  continue  its 
growth.  The  author  explains  that  "decision  making  is  what  you  do  when  you  don't 
know  what  to  do",  and  cites  evidence  that  people  are  not  good  natural  decision 
makers.  It  is  emphasized  that  decision  analysis  is  not  simply  a  logical  proce¬ 
dure,  but  also  an  "artistic"  process. 

Jin,  V.Y.  (1985,  May).  Delays  for  distributed  decisionmaking  organizations. 
Unpublished  thesis,  Massachusetts  Institute  of  Technology,  Cambridge, 
Massachusetts. 

In  this  thesis  an  algorithm  for  computing  time  delay  in  the  operation  of 
distributed  decision  making  systems  (DDM)  is  produced.  Interactors  in  DDM 
organizations  are  investigated  and  asynchronous  protocols  that  enrich 
performance  are  constructed. 

Johnson,  L.C.,  &  Mai,  N.  (1979)*  Decomposition  techniques:  linear  vs.  non¬ 
linear  models.  Organizational  Behavior  and  Human  Performance,  24,  60-66. 

Discriminates  between  the  scale  quality  in  a  simple  linear  model  and  the 
scale  quality  of  a  nonlinear  model  through  use  of  a  distortion  index.  This 
index  is  tested  in  a  simulation  study,  indicating  that  in  some  cases  the  linear 
model  can  misleadingly  disconnect  alternatives.  The  correlation  approach  does 
not  take  into  account  the  changes  which  may  occur  in  the  scale  upon  which 
alternatives  are  examined,  which  is  why  the  simple  linear  model  in  proposed. 

Johnson,  P.E.  (1980,  June,  in  press).  Cognitive  models  of  medical  problem 
solvers.  In  D.  Connelly,  E.  Benson,  &  D.  Burke  (Eds.),  Clinical  Decision  Mak¬ 
ing  and  Laboratory  Use.  Minneapolis,  Minnesota:  University  of  Minnesota 
Press. 

In  this  discussion  of  cognitive  models  of  medical  problem  solvers  the 
principal  theme  is  that  similarities  outweigh  differences  when  comparing 
individual  problem  solvers.  A  diagnostic  reasoning  model  of  physician 
knowledge  is  proposed  which  depicts  what  patients  with  certain  diseases  or 
conditions  should  "look  like". 

Johnson- Laird ,  P.N.  (1983).  Mental  models,  towards  a  cognitive  science  of 
language,  inference,  and  consciousness.  Cambridge,  Massachusetts:  Harvard 
University  Press. 

This  chapter  from  Johnson-Laird ' s  book  defines  "mental  models",  explains 
how  they  are  constructed,  and  interprets  them.  The  fhoory  itself  is  designed 
to  explain  the  higher  processes  of  cognition,  particularly  comprehension  and 
inference. 
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Jungermann,  H.  ( 1980a).  Speculations  about  decision- theoretic  aids  for  per¬ 
sonal  decisionmaking.  Acta  Psychologica,  45.,  7-34. 

Deals  with  techniques  which  assist  people  in  making  personal  decisions. 
Personal  decisions  are  defined  as  those  where  the  person  making  the  decision 
must  face  the  ramifications  of  his  actions.  Even  though  this  distinction  may 
seem  blurred  between  those  persons  making  decisions  for  themselves  versus  oth¬ 
ers,  the  author  holds  the  opinion  that  not  enough  attention  in  applied  decision 
research  is  given  to  personal  decisionmaking.  Structuring  the  problem,  as¬ 
sessing  ones  value  system,  and  examining  alternative  options  are  addressed  in 
helping  one  to  arrive  at  a  decision. 

Jungermann,  H.  ( 1980b,  September).  Structural  modeling  of  decision  problems. 
Paper  presented  at  the  88th  Annual  Convention  of  the  American  Psychological 
Association  (APA),  Montreal. 

The  author's  interest  lies  with  the  first  stage  in  the  process  of  decision 
making,  in  which  the  problem  is  represented  (translated  cognitively  into  a 
structure  that  represents  the  situation) .  It  is  proposed  that  the  questions 
one  asks  influence  the  answers  one  gets,  i.e.,  various  methods  of  posing  a 
decision  problem  to  a  person  stimulates  him  to  "reference  structure"  different 
sets  of  knowledge.  Examples  are  given  of  the  phases  of  the  decision  making 
process,  the  structuring  of  knowledge,  and  fault  trees. 

Kassirer,  J.P.,  &  Gorry,  G.A.  (1978).  Clinical  problem  solving:  a  behavioral 
analysis.  AnnalB  of  Internal  Medicine,  89,  245-255* 

Reports  on  some  of  the  specific  tactics  and  overall  strategies  composing 
the  problem-solving  behavior  of  trained  clinicians  as  they  were  "taking  the 
history  of  the  present  illness."  The  tape  recorded  behavior  of  this  inter¬ 
change  between  a  clinician  and  simulated  patient  was  analyzed.  The  authors 
suggest  that  the  information  has  potential  value  in  the  field  of  medical  educa¬ 
tion  and  in  devising  computer  programs  to  assume  the  process  of  diagnosis. 

Klayman,  J.  (1982).  Simulations  of  six  decision  strategies:  comparisons  of 
search  patterns,  processing  characteristics,  and  response  to  task  complexity 
(Working  paper  #1 00 ) . Chicago,  Illinois: University  of  Chicago. 

This  paper  addresses  questions  pertaining  to  the  modifications  made  in 
strategies  in  response  to  task  complexity  by  decision  makers.  A  primary  goal 
of  these  simulation  studies  entails  identifying  different  decii  m  strategies 
cn  the  basis  of  information  search  patterns.  It  is  reasoned  that  simulation 
can  be  very  useful  in  generating,  testing,  and  modifying  hypotheses  about  human 
decision  strategies  but  care  must  be  taken  to  insure  that  the  design  is  repre¬ 
sentative  of  the  impending  specific  decision  task. 

Klein,  G.A.  (1980,  October).  Automated  aids  for  the  proficient  decision  maker. 
Proceedings  of  the  Smith  Conference  on  Cybernetics  and  Society. 

Proposes  a  comparison-guided  model  of  decision-making,  maintaining  that 
proficient  decision  makers  are  able  to  recognize  and  subsequently  place  new 
situations  parallel  to  ones  previously  experienced.  The  types  of  decisions 
discussed  are  complex,  those  with  unusual  conditions.  It  is  asserted  that  a 
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redefinition  of  the  problem  by  the  expert  will  allow  utilization  of  all  re¬ 
sources  which  may  yield  a  solution  to  the  problem,  rather  than  using  a  decision 
aid  requiring  "rule- following".  The  author  believes  that  proficiency  is  not 
just  following  rules  and  procedures. 

Klein,  G.A.  (1981,  March).  A  perceptual/recognitional  model  of  decision  mak¬ 
ing.  Yellow  Springs,  Ohio:  Klein  Associates. 

Defends  the  view  that  skilled  decision  making  depends  on  perceptual  learn¬ 
ing  and  recognitional  capacities,  known  as  the  perceptual/recognitional  model. 
Argues  that  the  procedural  model,  wher^oy  one  follows  rules  and  procedures,  is 
not  adequate  for  use  in  making  comple>  decisions.  The  author  contends  that 
decision  making  can  be  treated  as  a  skill.  The  perceptual/recognitional  model 
proposes  the  use  of  analogous  experiences  to  form  judgments  and  requires  the 
evaluation  of  the  intangible  differences  between  experts  and  novices.  Suggests 
that  future  decision  aids  should  include  a  method  by  which  relevant  analogues 
may  be  identified. 

Kunreuther,  H.C.,  &  Schoemaker,  P.J.H.  (1979).  Decision  analysis  for  complex 
systems,  integrating  descriptive  and  prescriptive  components.  Knowledge: 
Creation,  Diffusion,  and  Utilization,  _2(3)»  389-412. 

A  framework  for  decision  analysis  in  complex  social  systems  is  presented. 

Of  major  concern  in  this  article  are  three  components:  (a)  descriptive  analyses 
of  organizational  environments  and  decision  models;  (b)  prescriptive  analysis, 
including  decision  methodology  and  decision  theory;  and,  (c)  the  relationship 
between  descriptive  and  prescriptive  analyses.  The  importance  of  descriptive 
analyses  is  emphasized  in  organizational  decision  making. 

Lichtenstein,  S.,  Earle,  T.C.,  &  Slovic,  P.  (1975).  Cue  utilization  in  a  nu¬ 
merical  prediction  task.  Journal  of  Experimental  Psychology:  Human  Percep¬ 
tions  and  Performance,  1 04 ( 1 ) ,  77-83 . 

Research  results  in  this  studv  were  obtained  from  an  experiment  in  which 
forty  subjects  were  trained  to  make  numerical  predictions  of  a  criterion  from  a 
cue.  It  was  found  that  subjects  regressed  each  cue  and  subsequently  averaged 
the  regressed  values.  It  is  suggested  that  by  analyzing  the  heuristics  util¬ 
ized  by  subjects,  we  can  further  our  knowledge  of  human  decision  making. 

Morris,  P.A.  (1975,  February).  Modeling  experts.  Manuscript  submitted  for 
publication. 

Addresses  the  problem  of  expert  resolution.  A  model  is  presented  which 
allows  the  decision  maker  to  arrive  at  a  solution,  whether  it  is  from  a  single 
expert  or  a  group  of  experts.  The  term  "surrogate  prior"  is  defined  and  shown 
to  play  a  role  in  measuring  the  joint  information  contained  in  the  probability 
assessments  generated  by  a  panel  of  experts  within  the  case  of  the  multi-ex¬ 
pert.  This  modeling  approach  forms  the  basis  for  analytical  thinking  about  the 
interaction  between  experts  and  the  manner  in  which  probabilities  should  be 
derived  from  experts. 
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Newman,  J.R.  (1977,  August).  Differential  weighting  for  prediction  and  deci¬ 
sion  making  studies:  a  study  of  ridge  regression  (SSRI  Research  Report 
77-1 ) . University  of  Southern  California: Social  Science  Research  Institute. 

This  study  describes  the  conditions  which  are  appropriate  for  differential 
or  simple  unit  weighting  of  predictor  variables  in  prediction  and/or  decision 
studies.  A  regression  model,  ridge  analysis  (RIDGE),  is  proposed  as  a  substi¬ 
tution  for  the  ordinary  least  squares  (OLS)  regression  analysis  because  of 
potential  difficulties  with  OLS  analysis.  The  RIDGE  method  of  estimation  may 
also  be  used  as  a  substitute  for  cross  validation. 

Norman,  D.A.  ( 1981a,  January).  Categorization  of  action  slips.  Psychological 
Review,  88(1 ) ,  1-15. 

The  author  outlines  a  system  of  activated  schemas  with  a  triggering  mecha¬ 
nism  which  determines  appropriate  time  for  activation,  which  is  a  satisfactory 
framework  for  categorization  and  analysis  of  action  slips.  This  theory  permits 
slips  to  be  categorized  into  three  areas:  (a)  errors  in  the  formation  of  the 
intention;  (b)  faulty  activation  of  schemas;  and,  (c)  faulty  triggering  of 
slips. 

Norman,  D.A.  ( 1981b).  A  psychologist  views  human  processing:  human  errors  and 
other  phenomena  suggest  processing  mechanisms.  Paper  presented  at:  IJCAI, 
Vancouver,  British  Columbia. 

Discusses  human  processing  structures  by  arguing  for  a  different  kind  of 
processing  mechanism  than  that  which  is  most  commonly  referred  to  ii  Artifi¬ 
cial  Intelligence.  The  form  of  human  error  is  of  major  importance  in  this 
study.  A  distinction  is  made  between  two  major  classes  of  errors,  mistakes 
versus  slips,  and  these  two  types  are  examined. 

Pitz,  G.F.  (1970).  On  the  processing  of  information:  probabilistic  and  other¬ 
wise.  In  G.  de  Zeeuw  et  al.  (Eds.),  Acta  Psychologies  54  Subjective  Probabil¬ 
ity  (pp.  201-213)*  Amsterdam:  North-Holland  Publishing  Company. 

Advocated  is  the  study  of  processes  such  as  attention,  encoding,  and  short¬ 
term  memory  in  relation  to  probabilistic  information  processing  (PIP)  in  order 
to  expand  our  knowledge  of  decision  making  behavior.  It  is  stressed  that  the 
information,  not  the  processing  itself,  is  probabilistic  in  nature.  Two  models 
of  opinion  revision  are  reviewed  which  incorporate  the  use  of  Bayes'  theorem  in 
their  models,  but  recognize  human  error.  Outlined  is  a  PIP  model  computer 
program. 

Rouse,  W.B.  (1983).  Models  of  human  problem  solving:  detection,  diagnosis, 
and  compensation  for  system  failures.  Automatica,  19(6),  613-625. 

Covers  models  of  human  problem  solving  with  an  emphasis  on  those  models 
which  predict  human  behavior  and  performance  to  support  design  and  evaluation. 
Areas  which  are  of  particular  concern  in  this  article  are  detection,  diagnosis, 
and  compensation  of  system  failures.  Proposed  is  an  outline  of  a  model  which 
may  influence  the  integration  and  progress  of  understanding  human  problem  solv¬ 
ing. 


11 


Saleh,  J. ,  Leal,  A.,  Kim,  J. ,  &  Pearl,  J.  (1979»  October).  Progress  toward  a 
goal-directed  decision  support  system  (Technical  Report  UCLA-ENG-CSL-7975) • 
University  of  California,  Los  Angeles:  School  of  Engineering  and  Applied 
Science. 

A  computerized  decision-support  system  is  described  which  was  developed  by 
the  authors  as  an  effective  replacement  for  decision-tret  representation.  They 
state  that  the  approach  is  preferable  because  of  the  clarity  and  purposefulness 
it  provides.  This  goal-directed  method  focuses  on  a  single  objective  at  any 
given  time  and  instructs  the  user  to  ignore  all  side  effects.  An  extensive 
amount  of  graphs,  structures,  and  representations  are  provided. 

Schoemaker,  P.J.H.  (1981 ).  Behavioral  issues  in  multiattribute  utility  model¬ 
ing  and  decision  analysis.  From  J.N.  Morse  (Ed.),  Organizations:  Multiple 
Agents  with  Multiple  Criteria.  Heidelberg,  Germany:  Springe r-Verlag. 

Various  empirical  issues  pertaining  to  decision  analysis,  specifically 
multiattribute  utility  (MAU)  modeling,  are  investigated.  The  issues  primarily 
concern  the  following  issues:  (a)  the  need  for  empirical  validations  in  norm¬ 
ative  decision  theory;  (b)  the  examination  of  the  intrinsic  characteristics  of 
MAU  preference  functions,  expressly  mathematical  complexity;  (c)  the  review  of 
method  bias;  and,  (d)  analysis  of  the  meanings  of  relative  importance  judg¬ 
ments. 

Schonemann,  P.H. ,  Cafferty,  T. ,  &  Rotton,  J.  (1975).  A  note  on  additive  func¬ 
tional  measurement.  Psychological  Review,  80(l),  85-87. 

In  this  report  it  is  demonstrated  that  the  additive  functional  measurement 
model  reduces  to  a  distinct  case  of  additive  conjoint  measurement.  Disputes 
N.  Anderson's  (1970)  functional  measurement  theory. 

Shanteau,  J.  (1975).  Averaging  versus  multiplying  combination  rules  of  infer¬ 
ence  judgment.  Acta  Psychologies,  59 ,  85-89. 

Two  combination  rules,  averaging  and  multiplying,  which  describe  sequential 
inference  judgments,  were  compared  through  three  experiments  in  this  study. 

Both  of  these  rules  depict  how  successive  pieces  of  evidence  are  combined  with 
previous  judgments  to  form  new  judgments.  Usage  of  the  averaging  rule  was  sup¬ 
ported  but  the  descriptive  ability  of  the  Bayesian  approach  (and  other 
approaches  which  use  the  multiplying  rule)  was  questioned.  It  may  be  inferred 
from  the  averaging  result  that  judgments  may  incorporate  processing  rules  dif¬ 
ferently  than  previously  speculated. 

Shanteau,  J.  (1985b,  September).  Psychological  characteristics  of  expert 
decision  makers  (Psychology  Report  #85-2). Manhattan,  Kansas:  Kansas  State 
University. 

This  study  investigates  personal  characteristics  in  expert  decision  makers, 
the  psychological  properties  which  the  author  feels  are  neglected  in  expert 
judgment  research.  Observations  are  made  concerning  these  characteristics  and 
some  unexplored  questions  about  expert  decision  makers  are  asked. 
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Shanteau,  J. ,  &  Gaeth,  G.J.  (1983,  January).  Training  expert  decision  makers 
to  ignore  irrelevant  information  (Human  Information  Processing  Report  No. 

82-5).  Manhattan,  Kansas:  Kansas  State  University. 

A  series  of  research  projects  were  performed  with  the  goal  of  revealing  the 
influence  of  irrelevant  information  on  expert  decision  makers.  Training  pro¬ 
grams  to  overcome  this  effect  were  discussed  and  the  long-term  effects  of  the 
training  were  evaluated. 

Shanteau,  J. ,  &  Nagy,  G.F.  (1984).  Information  integration  in  person  percep¬ 
tion:  theory  and  application.  In  M.  Cook  (Ed.),  Issues  in  person  perception 
(pp.  48-86).  London:  Methuen. 

This  selected  chapter  covers  some  of  the  principal  findings  from  research 
conducted  on  the  person  perception  model  with  the  information  integration  the¬ 
ory  (ITT).  Five  research  studies  are  alluded  to  which  exemplify  the  early  de¬ 
velopment  and  basic  assumptions  of  ITT,  which  is  traced  to  impression 
formation.  Cases  are  cited  which  show  how  ITT  can  be  utilized  in  the  analysis 
of  socially  relevant  interpersonal  decisions,  and  also  in  real-world  concerns. 

Simon,  H.A.  (1979,  Autumn).  On  how  to  decide  what  to  do.  Reprinted  from  The 
Bell  Journal  of  Economics,  j}(2),  494-507. 

This  article  provides  support  for  the  idea  that  economic  analysis  should 
not  simply  be  concerned  with  substantive  rationality,  but  also  with  the  proce¬ 
dural  aspects  of  decision  making.  The  author  discusses  approaches  to  proce¬ 
dural  rationality  within  the  fields  of  operations  research  and  management 
science,  artificial  intelligence,  computational  complexity,  and  cognitive  simu¬ 
lation. 

Slovic,  P. ,  Fischhoff,  B. ,  &  Lichtenstein,  S.  (1977).  Behavioral  decision 
theory.  Ann.  Rev.  Psychol. ,  1-39. 

The  author  focuses  on  the  psychological  aspects  of  individual  judgment  and 
decision  making  by  reviewing  descriptive  research  and  examining  decision  aids. 

A  survey  of  publications  (the  majority  of  them  dated  1971-75)  on  the  subject  of 
behavioral  decision  theory  is  provided  and  descriptive  studies  of  judgment  are 
evaluated.  An  attempt  is  made  to  enumerate  the  errors  in  decision-making  that 
may  arise  from  judgment  biases.  Some  areas  mentioned  under  the  topic  of  de¬ 
scriptive  research  include  probabilistic  judgment,  choice,  models  of  risky 
choice,  and  regression  approaches.  An  insightful  discussion  of  decision  aids, 
MAUT  research,  decision  analysis,  man/machine  systems,  and  use  of  decision  aids 
clarifies:  (a)  issues  which  a  decision  maker  may  face,  and  (b)  the  systems 
used  to  solve  problems. 

Slovic,  P. ,  &  Tversky,  A.  (1974,  February).  Who  accepts  Savage's  Axiom?  (ONR 
Technical  Report  NR  197-026).  Eugene,  Oregon:  Oregon  Research  Institute. 

Argues  that  the  established  principle:  "rational  choice  is  comparable  to 
logic  in  that  a  reasonable  person  who  understands  the  postulates  would  not  want 
to  violate  them"  is  open  to  doubt.  An  experiment  is  conducted  which  examines 
Savage's  independence  principle,  a  key  axiom  inherent  in  expected  utility  the¬ 
ory.  Discussed  is  the  difficulty  in  discriminating  between  rejection  of  a 
decision  principle  and  unsuccessful  "understanding"  of  it. 
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Stevens,  A.,  Collins,  A.,  &  Goldin,  S.E.  (1979)*  Misconceptions  in  student's 
understanding.  Int.  J.  Man-Machine  Studies,  1 1  ,  145-156. 

Proposed  in  this  research  report  is  a  system  which  tutors  students  with 
intelligent  computer  aided  instruction  (ICAI)  by  diagnosing  and  rectifying 
conceptual  misunderstandings.  A  theory  based  on  examination  of  tutoring  dia¬ 
logs  is  the  basis  for  this  system.  The  goal  structure  of  a  tutor  and  types  of 
conceptual  bugs  are  specified. 

Taylor,  E.N.  (1983,  September).  A  review  of  literature  relevant  to  unaided 
tactical  decision  making  (API  Research  Note  83-35).  Alexandria,  Virginia: 

Human  Resources  Research  Organization  (HumRRO). 

Provides  a  summary  of  literature  which  may  be  useful  in  developing  and  im¬ 
plementing  training  programs  pertaining  to  tactical  decision  making.  The  au¬ 
thors  contend  that  a  cause  for  concern  in  the  military  is  the  question  of 
whether  or  not  decision  makers  are  deciding  upon  the  best  courses  of  action  in 
rapidly  changing  situations  based  on  the  techniques  utilized  by  them  to  organ¬ 
ize,  analyze,  and  present  information.  Various  models  of  decision  making, 
along  with  problems  in  training,  are  examined  with  the  purpose  of  developing  a 
methodology  for  preparing  military  leaders  to  make  tactical  decisions. 

v.  Winterfeldt,  D.  (1975,  August).  An  overview,  integration,  and  evaluation  of 
utility  theory  for  decision  analysis  (SSRI  Research  Report  75-9 )'«  Los 
Angeles,  California:  Social  Science  Research  Institute. 

Reviews  current  measurement  theories  on  utility  modeling  and  assessment. 
Written  particularly  for  decision  analysts  who  are  interested  in  these  theories 
for  deciphering  and  solving  real  world  decision  problems  which  are  complex  in 
nature.  This  paper  aims  to  relate  the  present  utility  measurement  theory 
with  decision  analytic  practice.  The  paper  tries  to  lessen  the  gap  between  the 
theory  and  practice  of  utility  measurement,  which  may  be  attributed  in  part  to 
the  difficulty  in  comprehension  of  the  mathematical  language  inherent  in  the 
utility  theories.  It  is  concluded  that  even  though  utility  theories  can  be 
beneficial  in  the  structuring  process  of  evaluation,  they  are  usually  too  com¬ 
plex  for  usage  in  real  world  preference  assessment. 


Uncertainty 

Beach,  L.R.  (1966,  March).  Accuracy  and  consistency  in  the  revision  of 
subjective  probabilities.  IEEE  Transactions  on  Human  Factors  in  Electronics,  1_ 

(1),  29-36.  '  . . 

This  report  investigates  the  consistency  with  which  subjective  probabili¬ 
ties  are  revised.  Four  experiments  were  performed  in  which  models  of  probabil¬ 
istic  concept  formation  studies  and  Bayesian  subjective  probability  revision 
studies  were  combined.  Results  showed  that  subjects’  revisions  of  their  sub¬ 
jective  probabilities  can  be  accurately  predicted  with  Bayes’  theorem. 
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Beach,  L.R.,  Townes,  B.D. ,  Campbell,  F.L. ,  &  Keating,  G.W.  (1976).  Developing 
and  testing  a  decision  aid  for  birth  planning  decisions.  Organizational 
Behavior  and  Human  Performance,  1 5,  99-116. 

A  decision  aid  is  promoted  which  allows  one  to  calculate  subjective  ex¬ 
pected  utilities  (SEUs)  for  having  a  (another)  child;  a  value  hierarchy  is 
used.  The  method  is  intended  to  assist  people  with  their  birth  planning  deci¬ 
sions  so  that  they  may  logically  and  sensibly  weigh  the  pros  and  cons.  Deci¬ 
sions  are  divided  into  "manageable  chunks"  and  the  relative  utility  of  each 
chunk  is  evaluated.  The  authors  affirm  that  through  the  development  of  this 
aid  they  developed  a  method  applicable  to  other  areas  besides  birth  planning. 

Benbasat,  I.  &  Taylor,  R.N.  (1982,  July/ August) .  Behavioral  aspects  of  infor¬ 
mation  processing  for  the  design  of  management  information  systems.  IEEE 
Transactions  on  Systems,  Man,  and  Cybernetics,  SMC-1 2(4),  439-450. 

Cites  various  models  of  information  processing  which  account  for  human 
thinking  and  attempts  to  cover  the  capabilities,  as  well  as  the  inadequacies, 
in  information  processing  present  in  the  literature.  The  authors  believe  that 
management  information  systems  (MIS)  can  be  radically  improved  by  understanding 
the  behavioral  processes  by  which  humans  process  information  and  make  choices. 
This  will  aid  in  eliminating  the  problem  of  "overloading"  the  human  user. 

Brown,  R.V.  (1974,  October).  Modeling  subsequent  acts  for  decision  analysis; 
a  heretical  view.  Paper  presented  at  International  Conference  of  ORSA/TIMS, 
Puerto  Rico. 

Addresses  problems  in  the  technology  of  decision  analysis,  namely  structur¬ 
ing  of  the  model  itself.  The  conventional  decision  theory  model  in  the  form  of 
a  decision  tree  is  referred  to  as  "preposterior"  analysis,  which  is  faulted  by 
the  author  as  rarely  meeting  "real  world"  decision  analyses  requirements. 

This  could  possibly  be  a  reason  for  the  lack  of  success  with  decision  analysis 
in  information  decisions  as  opposed  to  successful  operating  decisions  which  are 
all  based  on  prior  information.  An  acts-as-events  model  is  proposed  which 
supposedly  provides  an  adequate  framework  for  assessment  (and  involves  a 
smaller  number  of  total  assessments)  and  allows  events  to  be  more  readily  visu¬ 
alized. 

Edwards,  W.  (1975,  August).  Research  on  the  technology  of  inference  and  deci¬ 
sion  (Report  No.  001597-F).  Los  Angeles,  California:  Advanced  Research 
Projects  Agency. 

The  technology  of  inference  and  decision  is  the  topic  of  this  research  re¬ 
port.  The  focus  is  on  the  elicitation  of  subjective  probabilities,  multi-at¬ 
tribute  utility  theory,  and  error  in  decision  analysis.  Stressed  is  the 
necessity  of  prioritizing  the  structuring  of  the  problem  along  with  processing 
information  and  the  elicitation  of  probabilities/utilities,  respectively. 

Einhorn,  H.J.  (1980).  Overconfidence  in  judgment.  New  Directions  for  Method¬ 
ology  of  Social  and  Behavioral  Science,  _4,  1-16. 

Argues  that  the  methods  we  practice  to  fulfill  our  goals  depend  upon  un¬ 
aided  judgment  rather  than  scientific  methodology.  The  author  believes  that  we 
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continually  "judge  our  own  judgment,"  and  overconfidence  in  judgment  is  a  re¬ 
sult  of  the  manner  in  which  outcome  feedback  is  utilized  to  evaluate  and  learn 
about  judgmental  accuracy.  The  idea  of  outcome  irrelevant  learning  structures 
(OILS)  is  presented.  Discusses  how  our  ability  to  learn  about  judgment  can  be 
enhanced. 

Einhorn,  H.J.  &  Hogarth,  R.M.  (1978).  Confidence  in  judgment:  persistence  of 
the  illusion  of  validity.  Psychological  Review,  85(5)»  395-416. 

Responds  to  the  question  ''Why  is  there  a  substantial  difference  between  a 
person's  confidence  in  his  own  judgment  and  research  findings  on  the  fal¬ 
libility  of  human  judgment?"  by  exploring  the  structure  of  judgmental  tasks  as 
it  affects  outcomes.  The  manner  in  which  outcomes  are  interpreted  and  used  is 
discussed,  as  well  as  the  relationship  between  learning  and  experience.  Pre¬ 
sents  a  model  in  which  learning  and  maintaining  confidence  in  one's  own  judg¬ 
ment  is  developed. 

Einhorn,  H.J.  &  Hogarth,  R.M.  (1981).  Behavioral  decision  theory:  processes 
of  judgment  and  choice.  Ann.  Rev.  Psychol.,  32,  53-88. 

This  review  of  behavioral  decision  theory  literature  argues  that  the 
judgmental  biases  exhibited  in  laboratory  situations  may  be  functional  in  natu¬ 
ral  settings.  Areas  reviewed  include  attention,  memory,  learning,  feedback, 
cognitive  representation,  and  conflict.  Assuming  a  broad  perspective  in  the 
research  of  decision  making  may  yield  the  most  insightful  and  generalized  re¬ 
sults. 

Einhorn,  H.J. ,  &  Hogarth,  R.M.  (1985,  January).  Ambiguity  and  uncertainty  in 
probabilistic  inference.  Psychological  Review,  in  press. 

Presents  a  model  in  reference  to  the  role  of  ambiguity  and  uncertainty  in 
inferential  judgments.  The  authors  show  how  this  model  may  be  associated  with 
Keynes'  concept  of  the  weight  of  evidence,  the  non-additivity  of  complementary 
probabilities,  risky  choice,  and  current  work  on  cognitive  heuristics.  It  is 
proposed  that  reaching  a  compromise  between  "what  is"  and  "what  might  have 
been"  or  "what  might  be"  is  focal  to  inferences  under  ambiguity  and  uncert¬ 
ainty. 

Einhorn,  H.J.,  &  Schacht,  S.  (1977).  Decisions  based  on  fallible  clinical 
judgment.  To  appear  in  Kaplan,  M. ,  &  Schwartz,  S.  (Eds.),  Human  Judgment  and 
Decision  Processes:  Applications  in  Problem  Settings:  Academic  Press. 

Addresses  the  problem  of  a  low  accuracy  rate  in  clinical  judgment.  An  ana¬ 
lytical  framework  for  dealing  with  questions  pertaining  to  the  validity  of 
clinical  judgment  is  illustrated.  Some  of  the  effects  of  clinical  judgment  on 
decisions  that  resulted  because  of  a  lack  of  correlation  between  some  criterion 
and  the  judgment  is  also  reviewed.  The  authors  assert  that  their  results  are 
applicable  to  all  situations  where  judgments  lead  to  making  decisions. 
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Fischhoff,  B.  (1985,  Vinter).  Environmental  reporting:  what  to  ask  the 
experts.  The  Journalist,  11—15- 

This  study  of  environmental  reporting  focuses  on  the  difficulty  in  obtain¬ 
ing  complete,  relevant,  and  valid  environmental  stories.  The  author  cites  the 
major  problem  as  one  of  overconfidence  displayed  by  specialists  who  do  not 
realize  the  limits  to  their  own  understanding. 

Fischhoff,  B. ,  Slovic,  P. ,  &  Lichtenstein,  S.  (1978).  Fault  trees:  sensitiv¬ 
ity  of  estimated  failure  probabilities  to  problem  representation.  Journal  of 
Experimental  Psychology:  Human  Perception  and  Performance,  £(2),  330-344. 

This  study  analyzes  the  effects  of  varying  three  aspects  of  fault  tree 
structure  on  the  assessment  of  a  fault  tree  for  the  event  of  "a  car  failing  to 
start."  Results  confirmed  that:  (a)  People  were  insensitive  to  what  had  been 
omitted  from  the  fault  tree;  (b)  When  the  amount  of  detail  was  increased  this 
did  not  significantly  affect  perceptions;  and,  (c)  The  perceived  importance  of 
a  certain  branch  was  increased  by  presenting  it  as  two  separate  component 
branches. 

Fryback,  D.G.,  Goodman,  B.C.,  &  Edwards,  V.  (1973).  Choices  among  bets  by  Las 
Vegas  gamblers:  absolute  and  contextual  effects.  Journal  of  Experimental 
Psychology,  98(2),  271-278. 

In  this  study,  two  experiments  conducted  at  the  Four  Queens  Casino  in  Las 
Vegas  refute  prior  research  evidence  that  accounted  for  choices  among  gambles. 
The  Expected  Value  (EV)  normally  is  highly  correlated  with  preferences  among 
gamblers.  It  may  be  inferred  that  the  attraction  of  a  gamble  is  not  totally 
dependent  upon  the  characteristics  of  that  gamble.  The  results  suggest  that 
certain  concepts  can  not  be  overlooked  in  the  construction  of  theories  that 
endeavor  to  depict  human  decision  making;  these  are:  anchoring,  adaptation, 
assimilation,  and  contrast. 

Hogarth,  R.M.  (1982,  March).  On  the  surprise  and  delight  of  inconsistent  re¬ 
sponses.  In  R.  Hogarth  (Ed.),  New  Directions  for  Methodology  of  Social  and 
Behavioral  Science:  Questions  Framing  and  Response  Consistency  (pp.  91-103). 
San  Francisco:  Josey  Bass. 

The  author  begins  by  providing  the  reader  with  various  definitions  of  the 
word  inconsistency  and  relates  inconsistency  tc  the  concept  of  variability.  The 
goals  and  steps  of  practical  inference  are  discussed,  and  sources  of  consis¬ 
tency  and  inconsistency  in  the  process  of  formulating  hypotheses  are  cited. 

Hudgens,  G.A.,  &  Fatkin,  L.T.  (1986,  February).  Sex  differences  in  risk  tak¬ 
ing:  repeated  sessions  on  a  computer-simulated  task  (Technical  Memorandum 
2-86 ) . Aberdeen  Proving  Ground,  Maryland:  U.S.  Army  Human  Engineering  Labora¬ 
tory. 

A  computer- gene rated  task  and  a  controlled  task  were  used  in  two  experi¬ 
ments  to  examine  sex  differences  in  risk-taking  behavior.  Male  and  female 
participants  were  shown  a  video  display  of  simulated  probabilities  and  took 
risks.  The  results  concur  with  the  current  research  that  states  men  are  more 
inclined  to  take  risks  than  women  in  a  diversity  of  situations. 
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Lichtenstein,  S.,  Earle,  T.C.,  &  Slovic,  P.  (1975).  Cue  utilization  in  a  nu¬ 
merical  prediction  task.  Journal  of  Experimental  Psychology:  Human  Percep¬ 
tions  and  Performance,  104(1),  77-85. 

Research  results  in  this  study  were  obtained  from  an  experiment  in  which 
forty  subjects  were  trained  to  make  numerical  predictions  of  a  criterion  from  a 
cue.  It  was  found  that  subjects  regressed  each  cue  and  subsequently  averaged 
the  regressed  values.  It  is  suggested  that  by  analyzing  the  heuristics  util¬ 
ized  by  subjects,  we  can  further  our  knowledge  of  human  decision  making. 

Lichtenstein,  S. ,  &  Slovic,  P.  (1973).  Response-induced  reversals  of  pref¬ 
erence  in  gambling:  an  extended  replication  in  Las  Vegas.  Journal  of  Experi¬ 
mental  Psychology,  101 ( 1 ) ,  16-20. 

An  experiment  was  conducted  at  the  Four  Queens  Casino  in  Las  Vegas  to  de¬ 
termine  the  effects  of  response  mode  upon  information-processing  strategies 
applied  by  subjects  in  gambling  decisions.  Different  strategies  were  used  for 
choosing  among  pairs  of  bets  as  opposed  to  assigning  monetary  values  to  single 
bets.  Bias  was  found  to  occur  in  favorable  and  unfavorable  gambles  as  a  res.lt 
of  cue-response  compatibility. 

Lopes,  L.L.  (1981 ).  Notes,  comments,  and  new  findings:  decision  making  in  the 
short  run.  Journal  of  Experimental  Psychology:  Human  Learning  and  Memory, 
7(5),  377-385. 

Addresses  the  question  "Why  do  psychologists  in  general  and  decision 
experts  treat  gambling  so  differently  from  other  probability  distributions?". 
The  author  analyzes  and  questions  the  role  that  mathematical  expectation  has 
in  affecting  theories  of  rational  choice  and  decision  making. 

MacGregor,  D. ,  &  Slovic,  P.  (1985,  March).  Perceived  acceptability  of  risk 
analysis  as  a  decision-making  approach.  Eugene,  Oregon:  Decision  Research,  A 
Branch  of  Perceptronics. 

Investigates  a  methodology  for  understanding  how  people  evaluate  decision 
making  approaches  involving  health  and  safety  risk  factors.  Three  methods  for 
making  a  consumer  product  safety  decision  were  appraised  on  scales  relating  to 
their  perceived  acceptability,  logical  soundness,  completeness,  and  sensitivity 
to  moral  and  ethical  concerns.  An  effect  on  judgments  was  not  shown  from  hav¬ 
ing  knowledge  of  the  consequences.  An  implication  made  is  that  people's  accep¬ 
tance  of  risk  may  be  greatly  determined  by  the  manner  in  which  decision  risks 
are  made. 

Morris,  P.A.  (1975,  February).  Modeling  experts.  Manuscript  submitted  for 
publication. 

Addresses  the  problem  of  expert  resolution.  A  model  is  presented  which 
allows  the  decision  maker  to  arrive  at  a  solution,  whether  it  is  from  a  single 
expert  or  a  group  of  experts.  The  term  "surrogate  prior"  is  defined  and  shown 
to  play  a  role  in  measuring  the  joint  information  contained  in  the  probability 
assessments  generated  by  a  panel  of  experts  within  the  case  of  the  multi-ex¬ 
pert.  This  modeling  approach  forms  the  basis  for  analytical  thinking  about  the 
interaction  between  experts  and  the  manner  in  which  probabilities  should  be 
derived  from  experts. 
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Newman,  J.R.  (1977,  August).  Differential  weighting  for  prediction  and  deci¬ 
sion  making  studies:  a  study  of  ridge  regression  (SSRI  Research  Report 
77-1 ) . University  of  Southern  California: Social  Science  Research  Institute. 

This  study  describes  the  conditions  which  are  appropriate  for  differential 
or  simple  unit  weighting  of  predictor  variables  in  prediction  and/or  decision 
studies.  A  regression  model,  ridge  analysis  (RIDGE),  is  proposed  as  a  substi¬ 
tution  for  the  ordinary  least  squares  (OLS)  regression  analysis  because  of 
potential  difficulties  with  OLS  analysis.  The  RIDGE  method  of  estimation  may 
also  be  used  as  a  substitute  for  cross  validation. 

Pitz,  G.F.  (1970).  On  the  processing  of  information:  probabilistic  and  other¬ 
wise.  In  G.  de  Zeeuw  et  al.  (Eds.),  Acta  Psychologica  34  Subjective  Probabil¬ 
ity  (pp.  201 -21 3).  Amsterdam:  North-Holland  Publishing  Company. 

Advocated  is  the  study  of  processes  such  as  attention,  encoding,  and  short¬ 
term  memory  in  relation  to  probabilistic  information  processing  (PIP)  in  order 
to  expand  our  knowledge  of  decision  making  behavior.  It  is  stressed  that  the 
information,  not  the  processing  itself,  is  probabilistic  in  nature.  Two  models 
of  opinion  revision  are  reviewed  which  incorporate  the  use  of  Bayes'  theorem  in 
their  models,  but  recognize  human  error.  Outlined  is  a  PIP  model  computer 
program. 

Shanteau,  P.  (1974).  Component  processes  in  risky  decision  making.  Journal  of 
Experimental  Psychology,  103(4),  680-691. 

The  goal  of  this  research  study  was  to  utilize  the  theory  of  information 
integration  by  applying  an  information  processing  approach  to  risky  decision 
judgments.  The  study  challenges  the  utility  theory  as  used  in  risky  decision 
making.  The  multiplying  model  is  advocated,  which  is  supported  by  data  result¬ 
ing  from  two  experiments. 

Slovic,  P.  Judgment,  choice,  and  societal  risk  taking.  In  K.R.  Hammond  (Ed.), 
Judgment  and  Choice  in  Public  Policy  Decisions.  AAAS  Symposium  Volume:  in 
press. 

The  author  senses  that  people  are  inadequately  equipped  to  make  societal 
decisions  in  our  state  of  rapid  technological  development.  This  article  deals 
with  the  determinants  of  perceived  and  acceptable  risk  within  the  context  of 
risk  assessment,  which  was  developed  as  a  result  of  society  attempting  to 
confront  problems  of  risk. 

Slovic,  P.  (1971,  April).  Information  processing,  situation  specificity,  and 
the  generality  of  risk-taking  behavior.  ORI  Research  Bulletin,  1 1 (3) ,  1-22. 

Inter-task  consistency  of  individual  differences  was  ascertained  in  two 
structurally  comparable  risk-taking  tasks.  A  relationship  was  shown  between 
information  processing  and  situational  specificity.  As  a  result  of  the  experi¬ 
ment,  it  was  determined  that  the  two  response  modes  triggered  different  methods 
of  processing  information  pertaining  to  probabilities  and  payoffs  in  a  way  that 
influenced  individual  differences  and  reduced  inter-task  consistencies.  The 
implication  is  that  it  is  improbable  to  find  significant  correlations  between 
risk-taking  measures  in  structurally  different  settings  or  between  risk-taking 
and  different  behaviors. 
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Slovic,  P. ,  Fischhoff,  B. ,  &  Lichtenstein,  S.  (1977).  Behavioral  decision 
theory.  Ann.  Rev.  Psychol.,  1-39. 

The  author  focuses  on  the  psychological  aspects  of  individual  judgment  and 
decision  making  by  reviewing  descriptive  research  and  examining  decision  aids. 

A  survey  of  publications  (the  majority  of  them  dated  1971-75)  on  the  subject  of 
Behavioral  Decision  Theory  is  provided  and  descriptive  studies  of  judgment  are 
evaluated.  An  attempt  is  made  to  enumerate  the  errors  in  decision-making  that 
may  arise  from  judgment  biases.  Some  areas  mentioned  under  the  topic  of  de¬ 
scriptive  research  include  probabilistic  judgment,  choice,  models  of  risky 
choice,  and  regression  approaches.  An  insightful  discussion  of  decision  aids, 
MAUT  research,  decision  analysis,  man/machine  systems,  and  use  of  decision  aids 
clarifies:  (a)  issues  which  a  decision  maker  may  face,  and  (b)  the  systems 

used  to  solve  problems. 

Slovic,  P. ,  Fischhoff,  B. ,  &  Lichtenstein,  S.  (1983,  December).  The  psycho¬ 
metric  study  of  risk  perception.  Eugene,  Oregon:  Decision  Research,  A  Branch 
of  Perceptronics. 

Deals  with  the  attempt  by  researchers  to  determine  what  the  term  "risky" 
means  to  people.  Factors  which  affect  the  response  to  perceived  risk  are  exam¬ 
ined,  with  the  goal  of  providing  more  effective  communication  risk  information, 
altering  people's  perceptions,  and  enhancing  management  strategy.  Reveiwed  are 
some  results  obtained  from  psychometric  studies  of  risk  perception. 

Slovic,  P. ,  &  Lichtenstein,  S.  (1968,  November).  Relative  importance  of 
probabilities  and  payoffs  in  risk  taking.  Journal  of  Experimental  Psychology 
Monograph,  78(3) ,  1-18. 

This  report  examines  the  beliefs  held  by  people  about  the  relevant  impor¬ 
tance  of  probabilities  and  payoffs  and  their  capability  to  act  on  the  premise 
of  these  beliefs  when  processing  information  included  in  the  description  of  a 
gamble.  Two  experiments  are  described  which  suggest  that  there  is  utility  in 
recognizing  decisions  about  gambling  within  the  context  of  information  process¬ 
ing. 

Slovic,  P. ,  &  Tversky,  A.  (1974,  February).  Who  accepts  Savage's  Axiom?  (ONR 
Technical  Report  NR  197-026).  Eugene,  Oregon:  Oregon  Research  Institute. 

Argues  that  the  established  principle:  "rational  choice  is  comparable  to 
logic  in  that  a  reasonable  person  who  understands  the  postulates  would  not  want 
to  violate  them"  is  open  to  doubt.  An  experiment  is  conducted  which  examines 
Savage's  independence  principle,  a  key  axiom  inherent  in  expected  utility  the¬ 
ory.  Discussed  is  the  difficulty  in  discriminating  between  rejection  of  a 
decision  principle  and  unsuccessful  "understanding"  of  it. 

Tversky,  A.,  &  Kahneman,  D.  (1973).  Availability:  a  heuristic  for  judging 
frequency  and  probability.  Cognitive  Psychology,  _5,  207-232. 

How  do  people  evaluate  the  frequency  of  classes  or  the  likelihood  of  events? 
According  to  this  article,  people  engage  in  using  a  limited  number  of  heu¬ 
ristics,  thus  diminishing  judgments  into  easier  ones.  A  judgmental  heuristic 
in  which  one  assesses  the  frequency  of  classes  or  probability  of  events  by 
availability  (the  quickness  with  which  relevant  instances  come  to  mind)  is 
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investigated.  It  is  believed  that  reliance  on  the  availability  or  representa¬ 
tive  heuristic  leads  to  systematic  biases.  By  analyzing  the  heuristics  that 
one  uses  in  judging  the  probability  of  an  event,  it  is  asserted  that  the  occur¬ 
rence  of  errors  in  human  judgment  under  uncertainty  may  be  decreased. 


Heuristics,  Biases,  and  Errors 

Beach,  L.R.  (1966,  March).  Accuracy  and  consistency  in  the  revision  of 
subjective  probabilities.  IEEE  Transactions  on  Human  Factors  in  Electronics,  7 
(1),  29-36.  . . .  " .  . .  " 

This  report  investigates  the  consistency  with  which  subjective  probabili¬ 
ties  are  revised.  Four  experiments  were  performed  in  which  models  of  probabil¬ 
istic  concept  formation  studies  and  Bayesian  subjective  probability  revision 
studies  were  combined.  Results  showed  that  subjects'  revisions  of  their  sub¬ 
jective  probabilities  can  be  accurately  predicted  with  Bayes'  theorem. 

Einhom,  H.J.,  &  Hogarth,  R.M.  (1978).  Confidence  in  judgment:  persistence  of 
the  illusion  of  validity.  Psychological  Review,  85(5),  395-416. 

Responds  to  the  question  "Why  is  there  a  substantial  difference  between  a 
person's  confidence  in  his  own  judgment  and  research  findings  on  the  fal¬ 
libility  of  human  judgment?"  by  exploring  the  structure  of  judgmental  tasks  as 
it  affects  outcomes.  The  manner  in  which  outcomes  are  interpreted  and  used  is 
discussed,  as  well  as  the  relationship  between  learning  and  experience.  Pre¬ 
sents  a  model  in  which  learning  and  maintaining  confidence  in  one’s  own  judg¬ 
ment  is  developed. 

Einhorn,  H.J. ,  &  Hogarth,  R.M.  (1981 ).  Behavioral  decision  theory:  processes 
of  judgment  and  choice.  Ann.  Rev.  Psychol.,  32,  53-88. 

This  review  of  behavioral  decision  theory  literature  argues  that  the 
judgmental  biases  exhibited  in  laboratory  situations  may  be  functional  in  natu¬ 
ral  settings.  Areas  reviewed  include  attention,  memory,  learning,  feedback, 
cognitive  representation,  and  conflict.  Assuming  a  broad  perspective  in  the 
research  of  decision  making  may  yield  the  most  insightful  and  generalized  re¬ 
sults. 

Einhorn,  H.J. ,  &  Hogarth,  R.M.  (1985,  January).  Ambiguity  and  uncertainty  in 
probabilistic  inference.  Psychological  Review,  in  press. 

Presents  a  model  in  reference  to  the  role  of  ambiguity  and  uncertainty  in 
inferential  judgments.  The  authors  show  how  this  model  may  be  associated  with 
Keynes'  concept  of  the  weight  of  evidence,  the  non-additivity  of  complementary 
probabilities,  risky  choice,  and  current  work  on  cognitive  heuristics.  It  is 
proposed  that  reaching  a  compromise  between  "what  is"  and  "what  might  have 
been"  or  "what  might  be"  is  focal  to  inferences  under  ambiguity  and  uncert¬ 
ainty. 
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Farquhar,  P.H.  (1982).  Utility  assessment  methods  (Working  Paper  81-5). 

Davis,  California:  University  of  California. 

Attempts  to  integrate  current  methods  for  evaluating  single-attribute  util¬ 
ity  functions.  Presents  various  new  methods  of  evaluation  which  may  be  benefi¬ 
cial  in  furthering  the  research  in  this  area,  emphasizing  the  varied  steps  in 
the  utility  assessment  process,  and  contrasting  methods  of  comparative  risk. 
Reviews  approximately  twenty-four  utility  assessment  methods. 

Fischhoff,  B. ,  &  Johnson,  S.  (1985,  March).  The  possibility  of  distributed 
decision  making.  Prepared  for  Workshop  on  Political-Military  Decision  Making, 
The  Hoover  Institution,  Stanford  University. 

Reviews  existing  problems  of  decision  making  distribution  when  dealing  with 
several  individuals  or  groups.  The  authors  propose  solutions  for  complications 
which  may  arise  as  a  result  of  using  these  systems,  and  discuss  design  and 
management  of  distributed  decision  making.  Outlined  are  possible  limitations 
which  could  affect  an  organization. 

Fryback,  D.G.,  Goodman,  B.C.,  &  Edwards,  W.  (1973).  Choices  among  bets  by  Las 
Vegas  gamblers:  absolute  and  contextual  effects.  Journal  of  Experimental 
Psychology,  98(2),  271-278. 

In  this  study,  two  experiments  conducted  at  the  Four  Queens  Casino  in  Las 
Vegas  refute  prior  research  evidence  that  accounted  for  choices  among  gambles. 
The  Expected  Value  (EV)  normally  is  highly  correlated  with  preferences  among 
gamblers.  It  may  be  inferred  that  the  attraction  of  a  gamble  is  not  totally 
dependent  upon  the  characteristics  of  that  gamble.  The  results  suggest  that 
certain  concepts  can  not  be  overlooked  in  the  construction  of  theories  that 
endeavor  to  depict  human  decision  making;  these  are:  anchoring,  adaptation, 
assimilation,  and  contrast. 

Goitein,  B.  (1984).  The  danger  of  disappearing  postdecision  surprise:  comment 
on  Harrison  and  March,  "decision  making  and  postdecision  surprises.”  Adminis¬ 
trative  Science  Quarterly,  29,  410-413. 

Comments  on  paper  written  by  Harrison  and  March  on  "Decisionmaking  and 
Postdecision  surprises"  (1984).  The  author  examines  the  issue  and  explains  why 
there  is  a  problem  with  the  disappearance  of  post  decision  surprise.  Believes 
that  "hindsight  bias,"  where  an  outcome  of  a  decision  may  be  seen  by  one  in 
hindsight  as  having  been  expected  to  a  greater  extent  than  it  actually  was,  is 
detrimental  because  it  prevents  individual  and  organizational  learning  about 
faults  in  the  decision  system.  Further  research  is  needed  on  how  to  reduce 
hindsight  bias. 

Halpin,  S.M.,  Johnson,  E.M.,  &  Thornberry,  J.A.  (1973,  August).  Cognitive 
reliability  in  manned  systems.  IEEE  Transactions  on  Reliability,  R-22(3) , 

1 65-170. 

Analyzes  cognitive  reliability  in  manned  systems,  which  the  authors  state 
is  an  intricate  function  of  attitudinal  and  structural  factors,  along  with 
their  interaction.  Cites  the  types  of  human  errors  which  may  transpire,  and 
discusses  factors  which  influence  the  chance  of  these  errors.  Due  to  the 
changing  role  of  man  in  complex  systems,  a  broadening  of  prevalent  approaches 
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to  human  performance  reliability  must  be  sought.  In  addition  to  describing  the 
tasks  which  are  becoming  progressively  more  important  in  man-machine  systems, 
the  term  "cognitive  reliability"  is  defined,  methods  of  classifying  cognitive 
errors  are  identified,  and  an  example  is  given  which  exemplifies  the  factors 
likely  to  affect  cognitive  errors. 

Klayman,  J.  (1982).  Simulations  of  six  decision  strategies:  comparisons  of 
search  patterns,  processing  characteristics,  and  response  to  task  complexity. 
(Working  paper  #100).  Chicago,  Illinois:  University  of  Chicago. 

This  paper  addresses  questions  pertaining  to  the  modifications  made  in 
strategies  in  response  to  task  complexity  by  decision  makers.  A  primary  goal 
of  these  simulation  studies  entails  identifying  different  decision  strategies 
on  the  basis  of  information  search  patterns.  It  is  reasoned  that  simulation 
can  be  very  useful  in  generating,  testing,  and  modifying  hypotheses  about  human 
decision  strategies  but  care  must  be  taken  to  insure  that  the  design  is  repre¬ 
sentative  of  the  impending  specific  decision  task. 

Klayman,  J.  (1983).  Analysis  of  predecisional  information  search  patterns. 

In  P.  Humphreys,  0.  Svenson,  &  A.  Vari  (Eds.),  Analyzing  and  Aiding  Decision 
Processes.  Amsterdam:  North-Holland. 

The  present  paper  discusses  the  analysis  of  information  gathering  patterns 
as  a  tool  for  process  tracing.  Attention  to  complex  and  diverse  decision 
approaches  has  underscored  the  necessity  of  developing  more  refined  process 
tracing  analyses  (analyzing  information  gathering  patterns).  Two  major  methods 
of  improving  search  analysis  are  examined.  An  experimental  example  using  these 
techniques  is  given,  and  applications  are  submitted  in  three  areas. 

Lichtenstein,  S.,  Earle,  T.C.,  &  Slovic,  P.  (1975).  Cue  utilization  in  a  nu¬ 
merical  prediction  task.  Journal  of  Experimental  Psychology:  Human  Percep¬ 
tions  and  Performance,  104(1 ),  77-85. 

Research  results  in  this  study  were  obtained  from  an  experiment  in  which 
forty  subjects  were  trained  to  make  numerical  predictions  of  a  criterion  from  a 
cue.  It  was  found  that  subjects  regressed  each  cue  and  subsequently  averaged 
the  regressed  values.  It  is  suggested  that  by  analyzing  the  heuristics  util¬ 
ized  by  subjects,  we  can  further  our  knowledge  of  human  decision  making. 

Lichtenstein,  S.,  <5  Slovic,  P.  ( 1 975 ) •  Response-induced  reversals  of  pref¬ 
erence  in  gambling:  an  extended  replication  in  Las  Vegas.  Journal  of  Experi¬ 
mental  Psychology,  1 01 ( 1 ) ,  16-20. 

An  experiment  was  conducted  at  the  Four  Queens  Casino  in  Las  Vegas  to  de¬ 
termine  the  effects  of  response  mode  upon  information-processing  strategies 
applied  by  subjects  in  gambling  decisions.  Different  strategies  were  used  for 
choosing  among  pairs  of  bets  as  opposed  to  assigning  monetary  values  to  single 
bets.  Bias  was  found  to  occur  in  favorable  and  unfavorable  gambles  as  a  result 
of  cue-response  compatibility. 
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Lopes,  L.L.  (1981).  Notes,  comments,  and  new  findings:  decision  making  in  the 
short  run.  Journal  of  Experimental  Psychology:  Human  Learning  and  Memory, 
7(5),  377-385. 

Addresses  the  question  "Why  do  psychologists  in  general  and  decision 
experts  treat  gambling  so  differently  from  other  probability  distributions?". 
The  author  analyzes  and  questions  the  role  that  mathematical  expectation  has 
in  affecting  theories  of  rational  choice  and  decision  making. 

Morier,  D.M,  &  Borgida,  E.  (1984,  June).  The  conjunction  fallacy:  a  task 
specific  phenomenon?  Personality  and  Social  Psychology  Bulletin,  10(2)  243- 
252.  '  ^ 

The  goal  of  this  study  was  to  ascertain  the  extent  to  which  the  conjunction 
fallacy,  a  type  of  judgmental  error,  reflects  task  specific  misunderstanding  of 
specific  judgment  problems.  Results  show  that:  (a)  Task  specificity  is  a 
factor  in  the  subjects'  misunderstanding  of  conjunction  problems,  and  (b)  an 
unprejudiced  approach  to  conjunction  fallacy  may  lower  but  not  eradicate  the 
conjunctive  error  rate  for  problems  which  are  not  depictive  of  representative 
thinking. 

Norman,  D.A.  (1980,  August).  Errors  in  human  performance  (Report  No.  8004). 

La  Jolla,  California:  Center  for  Human  Information  Processing. 

This  report's  area  of  concentration  lies  in  human  error,  particularly  error 
made  by  skilled  operators  in  complex,  demanding  systems.  Through  this  analysis 
of  human  error,  various  applications  of  human  information  processing  are  de¬ 
lineated  and  a  classification  scheme  for  errors  is  outlined.  Short-term  memory 
and  human  attentional  limitations  are  examined  in  the  design  of  human-machine 
interfaces. 

Norman,  D.A.  (1981  a,  January).  Categorization  of  action  slips.  Psychological 
Review,  88(l),  1-15. 

The  author  outlines  a  system  of  activated  schemas  with  a  triggering  mecha¬ 
nism  which  determines  appropriate  time  for  activation,  which  is  a  satisfactory 
framework  for  categorization  and  analysis  of  action  slips.  This  theory  permits 
slips  to  be  categorized  into  three  areas:  (a)  errors  in  the  formation  of  the 
intention;  (b)  faulty  activation  of  schemas;  and,  (c)  faulty  triggering  of 
slips. 

Norman,  D.A.  ( 1981b).  A  psychologist  views  human  processing:  human  errors  and 
other  phenomena  suggest  processing  mechanisms.  Paper  presented  at:  IJCAI, 
Vanco'iver,  British  Columbia. 

Discusses  human  processing  structures  by  arguing  for  a  different  kind  of 
processing  mechanism  than  that  which  is  most  commonly  referred  to  in  Artifi¬ 
cial  Intelligence.  The  form  of  human  error  is  of  major  importance  in  this 
study.  A  distinction  is  made  between  two  major  classes  of  errors,  mistakes 
versus  slips,  and  these  two  types  are  examined. 
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Norman,  D.A.  ( 1981c,  August).  Steps  toward  a  cognitive  engineering:  system 
images,  system  friendliness,  mental  models.  Paper  presented  at  Symposium  on 
Models  of  Human  Performance,  ONR  Contractor's  meeting,  La  Jolla,  California. 

Contends  that  current  inappropriate  system  design  does  not  allow  optimum 
efficacy  between  the  users  and  machines.  Through  the  study  of  human  error,  it 
is  discovered  that  human  usage  should  be  a  factor  warranting  primary  considera¬ 
tion  by  designers.  The  author  proposes  a  cohesive  system,  a  "friendly  system", 
composed  of  the  principles  underlying  both  cognitive  science  and  human  factors. 

Pitz,  G.F.  (1970).  On  the  processing  of  information:  probabilistic  and  other¬ 
wise.  In  G.  de  Zeeuw  et  al.  (Eds.),  Acta  Psychologica  34  Subjective  Probabil¬ 
ity  (pp.  201-213).  Amsterdam:  North-Holland  Publishing  Company. 

Advocated  is  the  study  of  processes  such  as  attention,  encoding,  and  short¬ 
term  memory  in  relation  to  probabilistic  information  processing  (PIP)  in  order 
to  expand  our  knowledge  of  decision  making  behavior.  It  is  stressed  that  the 
information,  not  the  processing  itself,  is  probabilistic  in  nature.  Two  models 
of  opinion  revision  are  reviewed  which  incorporate  the  use  of  Bayes'  theorem  in 
their  models,  but  recognize  human  error.  Outlined  is  a  PIP  model  computer 
program. 

Rouse,  W.B.  <5  Rouse,  S.H.  (1983»  July/ August) .  Analysis  and  classification  of 
human  error.  IEEE  Transactions  on  Systems,  Man,  and  Cybernetics,  SMC-13(4), 
539-549.  “  . . .  '  ^  . . 

This  issue  is  devoted  to  understanding  human  error.  It  is  defined  through 
the  causal  approach  as  error  that  is  seldom  random  and  traceable  to  causes, 
which  once  identified,  can  be  obliterated  or  lessened.  A  methodology  is  pre¬ 
sented  which  allows  one  to  analyze  and  classify  human  error;  this  is  exempli¬ 
fied  by  comparing  the  use  of  checklists  in  aviation.  The  goal  is  to  examine 
human  error  and  its  causes,  contributing  factors,  and  contributing  events. 

Schoemaker,  P.J.H.  (1981 ).  Behavioral  issues  in  multiattribute  utility  and 
modeling  and  decision  analysis.  From  J.N.  Morse  (Ed.),  Organizations:  Multi¬ 
ple  Agents  with  Multiple  Criteria.  Heidelberg,  German:  Springer-Verlag. 

Various  empirical  issues  pertaining  to  decision  analysis,  specifically 
multiattribute  utility  (MAU)  modeling,  are  investigated.  The  issues  primarily 
concern  the  following  issues:  (a)  the  need  for  empirical  validations  in  norm¬ 
ative  decision  theory;  (b)  the  examination  of  the  intrinsic  characteristics  of 
MAU  preference  functions,  expressly  mathematical  complexity;  (c)  the  review  of 
method  bias;  and,  (d)  analysis  of  the  meanings  of  relative  importance  judg¬ 
ments. 

Shanteau,  J.  (1975).  Averaging  versus  multiplying  combination  rules  of  infer¬ 
ence  judgment.  Acta  Psychologica,  39,  83-89. 

Two  combination  rules,  averaging  and  multiplying,  which  describe  sequential 
inference  judgments,  were  compared  through  three  experiments  in  this  study. 

Both  of  these  rules  depict  how  successive  pieces  of  evidence  are  combined  with 
previous  judgments  to  form  new  judgments.  Usage  of  the  averaging  rule  was  sup¬ 
ported  but  the  descriptive  ability  of  the  Bayesian  approach  (and  other 
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approaches  which  use  the  multiplying  rule)  was  questioned.  It  may  be  inferred 
from  the  averaging  result  that  judgments  may  incorporate  processing  rules  dif¬ 
ferently  than  previously  speculated. 

Slovic,  P.  Judgment,  choice,  and  societal  risk  taking.  In  K.R.  Hammond  (Ed.), 
Judgment  and  Choice  in  Public  Policy  Decisions.  MAS  Symposium  Volume:  in 
press. 

The  author  senses  that  people  are  inadequately  equipped  to  make  societal 
decisions  .In  our  state  of  rapid  technological  development.  This  article  deals 
with  the  determinants  of  perceived  and  acceptable  risk  within  the  context  of 
risk  assessment,  which  was  developed  as  a  result  of  society  attempting  to  con¬ 
front  problems  of  risk. 

Slovic,  P. ,  Fischhoff,  B. ,  &  Lichtenstein,  S.  (1977).  Behavioral  decision 
theory.  Ann.  Rev.  Psychol.,  1-39* 

The  author  focuses  on  the  psychological  aspects  of  individual  judgment  and 
decision  making  by  reviewing  descriptive  research  and  examining  decision  aids. 

A  survey  of  publications  (the  majority  of  them  dated  1971-75)  on  the  subject  of 
Behavioral  Decision  Theory  is  provided  and  descriptive  studies  of  judgment  are 
evaluated.  An  attempt  is  made  to  enumerate  the  errors  in  decision-making  that 
may  arise  from  judgment  biases.  Some  areas  mentioned  under  the  topic  of  de¬ 
scriptive  research  include  probabilistic  judgment,  choice,  models  of  risky 
choice,  and  regression  approaches.  An  insightful  discussion  of  decision  aids, 
MAUT  research,  decision  analysis,  man/machine  systems,  and  use  of  decision  aids 
clarifies:  (a)  issues  which  a  decision  maker  may  face,  and  (b)  the  systems 
used  to  solve  problems. 

Slovic,  P. ,  &  Tversky,  A.  (1974,  February).  Who  accepts  Savage's  Axiom?  (ONR 
Technical  Report  NR  197-026).  Eugene,  Oregon:  Oregon  Research  Institute. 

Argues  that  the  established  principle:  "rational  choice  is  comparable  to 
logic  in  that  a  reasonable  person  who  understands  the  postulates  would  not  want 
to  violate  them"  is  open  to  doubt.  An  experiment  is  conducted  which  examines 
Savage's  independence  principle,  a  key  axiom  inherent  in  expected  utility  the¬ 
ory.  Discussed  is  the  difficulty  in  discriminating  between  rejection  of  a 
decision  principle  and  unsuccessful  "understanding"  of  it. 

Tversky,  A.,  &  Kahneman,  D.  (1973)*  Availability:  a  heuristic  for  judging 
frequency  and  probability.  Cognitive  Psychology,  5.,  207-232. 

How  do  people  evaluate  the  frequency  of  classes  or  the  likelihood  of  events? 
According  to  this  article,  people  engage  in  using  a  limited  number  of  heu¬ 
ristics,  thus  diminishing  judgments  into  easier  ones.  A  judgmental  heuristic 
in  which  one  assesses  the  frequency  of  classes  or  probability  of  events  by 
availability  (the  quickness  with  which  relevant  instances  come  to  mind)  is 
investigated.  It  is  believed  that  reliance  on  the  availability  or  representa¬ 
tive  heuristic  leads  to  systematic  biases.  By  analyzing  the  heuristics  that 
one  uses  in  judging  the  probability  of  an  event,  it  is  asserted  that  the  occur¬ 
rence  of  errors  in  human  judgment  under  uncertainty  may  be  decreased. 
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Military  Decision  Making 


Burke,  J.T.  (1986,  March).  The  next  step:  managing  battle  with  technology. 
ARMY,  56-60. 

Discusses  the  concept  of  the  Battlefield  Management  System  (BMS)  and  advo¬ 
cates  its  role  in  improving  the  speed  and  effectiveness  of  combat  operations 
and  reducing  battlefield  stress.  Col.  (U.S.  Army  retired)  Burke  does  not  sug¬ 
gest  replacing  the  human  factor,  but  rather  applying  advanced  technology  to 
assist  soldiers,  leaders,  and  staff  people  in  coping  with  the  hectic  movement 
of  combat  action.  BMS  is  designed  with  the  purpose  of  allowing  humans  to  con¬ 
centrate  on  creative  decision  making  while  it  stores  and  processes  data  with 
great  speed.  BMS  would  be  an  immense  asset  to  the  battalion  S2  and  S3.  It  is 
argued  that  BMS  can  be  used  to  train  combat  crews  and  entire  units  through 
battlefield  simulation  and  will  save  millions  of  dollars,  a  substantial  amount 
of  time,  and  increase  efficiency  in  field  training. 

Cohen,  M.S.,  &  Freeling,  A.N.S.  (1981,  February).  The  impact  of  information  on 
decisions:  command  and  control  system  evaluation  (Technical  Report  '  -1 ). 

Falls  Church,  Virginia:  Decision  Science  Consortium,  Inc. 

This  study  applies  decision  analytic  concepts  to  the  problem  of  determining 
value  of  information  in  the  design  and  evaluation  of  information  systems. 
Although  many  current  systems  furnish  an  abundance  of  data,  the  decision  maker 
is  often  unable  to  pick  out  the  data  essential  to  the  problem.  Necessary  in¬ 
formation  is  that  which  has  a  high  probability  of  changing  a  decision  and  im¬ 
proving  decision  outcome.  This  study  modifies  standard  VOI  techniques  in  order 
to  simplify  them  and  encourage  their  usage. 

Fischhoff,  B.,  &  Johnson,  S.  (1985,  March).  The  possibility  of  distributed 
decision  making.  Prepared  for  Workshop  on  Political-Military  Decision  Making, 
The  Hoover  Institution,  Stanford  University. 

Reviews  existing  problems  of  decision  making  distribution  when  dealing  with 
several  individuals  or  groups.  The  authors  propose  solutions  for  complications 
which  may  arise  as  a  result  of  using  these  systems,  and  discuss  design  and 
management  of  distributed  decision  making.  Outlined  are  possible  limitations 
which  could  affect  an  organization. 

Jacobs,  T.O.  (1976,  September).  Analysis  and  evaluation  of  tactical  decision 
making.  Paper  presented  at  American  Psychological  Association  Meeting, 
Washington,  D.C. 

Outlines  a  study  in  which  an  illustration  of  how  incumbent  generals,  colo¬ 
nels,  and  lieutenant  colonels  define  command  and  control  performances.  Five 
general  categories  which  emerged  were:  getting  and  using  information,  plan¬ 
ning,  organizing,  controlling/directing,  and  leadership/personnel.  Reports  on 
an  experiment  in  which  the  usefulness  of  automating  analysis  and  production 
tasks  is  analyzed  and  proves  to  be  applicable  to  organizational  decision  mak¬ 
ing.  Among  the  findings  reported  are:  (a)  The  battalion  level  differs  from 
division  command  level  in  that  the  descriptive  model  suggests  a  "process-ori¬ 
ented  reactive  orientation"  for  battalion  level  decision  makers;  (b)  the  time 
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perspective  is  very  limited  for  the  battalion  commander  in  combat;  and,  (c)  ex¬ 
periments  in  the  area  of  tactical  decision  making  may  require  the  utilization 
of  real  incumbents  in  realistic  situations. 

Phelps,  R.H.,  Halpin,  S.M.,  &  Johnson,  E.M.  (1981,  January).  A  decision 
support  framework  for  decision  aid  designers  (ARI  Technical  Report  504). 
Alexandria,  Virginia:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

Describes  a  decision  support  framework  to  be  utilized  by  developers  of 
decision  aids  to  assist  them  in  the  organization  and  categorization  of  decision 
aiding  procedures.  This  framework  succinctly  distinguishes  between  two  catego¬ 
ries  of  aids  and  exemplifies  the  development  and  execution  of  the  aid  system. 
The  authors  suggest  that  utilization  of  this  framework  could  result  in  maximum 
productivity  of  their  systems.  Army  Intelligence  decision  making  examples  are 
cited  which  typify  the  components  of  the  framework.  This  decision  support 
framework  is  designed  to  represent  the  decision  support  components,  not  the 
decision  process  itself. 


O'Connor,  M.F.  (1977,  July).  Procedures  for  assessing  the  value  of  command  and 
control  capabilities  (Technical  Report  77-4).  McLean7  Virginia:  Decisions 
and  Designs,  Inc. 

Addresses  the  question:  How  much  command  and  control  (C2)  capability  is 
enough?  In  order  for  one  to  answer  this  question,  the  cost  of  achieving  that 
capability  must  be  weighed  against  the  potential  benefits  of  allocating  funds 
elsewhere.  A  value  is  assigned  to  the  action  of  deciding  to  deploy  a  certain 
C 2  system  in  a  precarious  future.  The  trade-offs  which  must  be  made  in  order 
to  resolve  this  question  are  analyzed.  The  paper  is  divided  into  four  sec¬ 
tions;  discussed  are:  (a)  the  theory  of  assessing  value  for  multiple  attrib¬ 
uted  alternatives;  (b)  the  use  of  scenarios  as  a  solution  to  the  problem  with 
decision  trees;  (c)  the  application  of  value  assessment  procedures;  and,  (d)  a 
specific  problem:  the  trade-offs  between  cobt  and  multiple  performance  meas¬ 
ures,  which  illustrates  the  use  of  utility  assessment  procedures  with  the  C2 
problem. 

Taylor,  E.N.  (1983,  September).  A  review  of  literature  relevant  to  unaided 
tactical  decision  making  (ARI  Research  Note  83-35)»  Alexandria,  Virginia: 

Human  Resources  Research  Organization  (HumRRO). 

Provides  a  summary  of  literature  which  may  be  useful  in  developing  and  im¬ 
plementing  training  programs  pertaining  to  tactical  decision  making.  The  au¬ 
thors  contend  that  a  cause  for  concern  in  the  military  is  the  question  of 
whether  or  not  decision  makers  are  deciding  upon  the  best  courses  of  action  in 
rapidly  changing  situations  based  on  the  techniques  utilized  by  them  to  organ¬ 
ize,  analyze,  and  present  information.  Various  models  of  decision  making, 
along  with  problems  in  training,  are  examined  with  the  purpose  of  developing  a 
methodology  for  preparing  military  leaders  to  make  tactical  decisions. 
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Methodology 


Klayman,  J.  (1982).  Simulations  of  six  decision  strategies:  comparisons  of 
search  patterns,  processing  characteristics,  and  response  to  task  complexity. 
(Working  paper  #100).  Chicago,  Illinois:  University  of  Chicago. 

This  paper  addresses  questions  pertaining  to  the  modifications  made  in 
strategies  in  response  to  task  complexity  by  decision  makers.  A  primary  goal 
of  these  simulation  studies  entails  identifying  different  decision  strategies 
on  the  basis  of  information  search  patterns.  It  is  reasoned  that  simulation 
can  be  very  useful  in  generating,  testing,  and  modifying  hypotheses  about  human 
decision  strategies  but  care  must  be  taken  to  insure  that  the  design  is  repre¬ 
sentative  of  the  impending  specific  decision  task. 

Klayman,  J.  (1983).  Analysis  of  predecisional  information  search  patterns. 

In  P.  Humphreys,  0.  Svenson,  &  A.  Vari  (Eds.),  Analyzing  and  Aiding  Decision 
Processes.  Amsterdam:  North-Holland. 

The  present  paper  discusses  the  analysis  of  information  gathering  patterns 
as  a  tool  for  process  tracing.  Attention  to  complex  and  diverse  decision 
approaches  has  underscored  the  necessity  of  developing  more  refined  process 
tracing  analyses  (analyzing  information  gathering  patterns).  Two  major  methods 
of  improving  search  analysis  are  examined.  An  experimental  example  using  these 
techniques  is  given,  and  applications  are  submitted  in  three  areas. 

Schweiger,  D.M.,  Anderson,  C.R.,  &  Locke,  E.A.  (1985).  Complex  decision  mak¬ 
ing:  a  longitudinal  study  of  process  and  performance.  Organizational  Behav¬ 
ior  and  Human  Decision  Processes,  36,  245-272. 

Simultaneous  verbal  protocols  were  employed  to  study  individual  decision 
making  processes.  The  authors  describe  the  protocol  analysis  procedure  used 
and  present  the  coding  scheme  developed  to  analyze  the  data.  Results  of  two 
experiments  show  that  thought  processes  used  in  complex  decision  making  are 
associated  with  subsequent  performance:  subjects  using  causal  analysis  per¬ 
formed  more  successfully  than  those  who  did  not. 
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Models 


Adelman,  L. ,  Donnell,  M.L. ,  Patterson,  J.F. ,  &  Weiss,  J.J.  (1981,  February). 
Issues  in  the  design  and  evaluation  of  decision-analytic  aids  (Technical 
Report  81-1-304). McLean,  Virginia:  Decisions  and  Designs,  Inc. 

Addresses  three  important  issues  which  are  relevant  to  the  design  and 
evaluation  of  decision  analytic  aids  and  presents  a  framework  which  illustrates 
the  decision  making  organization.  The  three  interfaces  identified  are:  be¬ 
tween  the  decision  aid  and  the  user,  between  the  user  and  the  extent  to  which 
the  decision  aid  facilitates  the  decision  making  processes  of  the  organization, 
and  between  the  decision  making  organization  and  whether  or  not  the  quality  of 
the  organization's  decision  making  has  improved  the  environment.  Without  the 
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smooth  functioning  of  the  three  interfaces,  effective  integration  of  decision 
aids  into  organizations  can  not  be  accomplished.  The  author  covers  some  ways 
to  attract  the  user  to  the  aid  and  to  simplify  its  overall  operation. 

Alter,  S.  (1977,  February).  Why  is  man-computer  interaction  important  for 
decision  support  systems?  Interfaces,  7.(2),  109-115. 

Argues  that  interactive,  conversational  decision  support  systems  do  not 
represent  their  true  identities  in  many  cases.  The  author  believes  that  better 
man-computer  interfaces  are  needed,  but  not  necessarily  by  managers  themselves. 
Suggests  that  the  focus  should  be  on  experts  who  understand  the  details  under¬ 
lying  a  support  system;  if  managers  do  not  learn  the  correct  usage  of  decision 
support  systems,  then  usage  should  be  discouraged  without  the  aid  of  an  inter¬ 
mediary. 

Andriole,  S.J.  (1982,  July/ August) .  The  design  of  microcomputer-based  personal 
decision  aiding  systems.  IEEE  Transactions  on  Systems,  Man,  and  Cybernetics, 
SMC-12(4),  463-469.  “  ~ 

Presents  an  approach  to  the  design  and  development  of  personal  computer- 
based  decision  aiding  systems  which  views  the  human  problem-solver  as  an  active 
user  who  must  be  able  to  use  the  computer  without  the  aid  of  a  problem-solving 
intermediary.  In  order  to  successfully  utilize  a  system,  the  author  states 
that  the  design  of  the  system  should  encompass  the  measurement  and  integration 
of  cognitive,  situational,  and  physiological  variables  which  should  inform  and 
alter  the  decision-aiding  process.  It  is  suggested  that  without  explicit  user 
requirements,  interactive  systems  fulfill  the  needs  of  the  designer  and  not  the 
target  user.  Problem  solvers  are  categorized  according  to  their  experience 
with  computer-based  systems  and  grouped  according  to  their  "cognitive  styles" 
in  order  to  formulate  computer-generated  information  and  strengthen  man's  in¬ 
teraction  with  the  computer.  The  user's  physiological  state  and  situational 
characteristics  are  examined  as  well. 

Baker,  J.D.  (1970).  Quantitative  modelling  of  human  performance  in  information 
systems.  Ergonomics,  13(6),  645-664. 

Describes  a  general  information  system  model,  MAN,  which  emphasizes  the 
importance  of  the  user  in  man-machine  systems.  The  ultimate  goal  is  to  acquire 
the  ability  to  utilize  the  results  of  human  information  processing  studies  to 
predict  the  impact  on  system  performance.  The  framework  structure  is  based  on 
the  supposition  that  there  are  five  central  operations  in  the  performance  of  an 
information  system:  screen,  transform,  input,  assimilate,  and  decide. 

Benbasat,  I.  &  Taylor,  R.N.  (1982,  July/ August) .  Behavioral  aspects  of  infor¬ 
mation  processing  for  the  design  of  management  information  systems.  IEEE 
Transactions  on  Systems,  Man,  and  Cybernetics,  SMC-12(4) ,  439-450. 

Cites  various  models  of  information  processing  which  account  for  human 
thinking  and  attempts  to  cover  the  capabilities,  as  well  as  the  inadequacies, 
in  information  processing  present  in  the  literature.  The  authors  believe  that 
management  information  systems  (MIS)  can  be  radically  improved  by  understanding 
the  behavioral  processes  by  which  humans  process  information  and  make  choices. 
This  will  aid  in  eliminating  the  problem  of  "overloading"  the  human  user. 
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Benel,  D.C.R.,  A  Pain,  R.F.  (1985).  The  human  factors  usability  laboratory  in 
product  evaluation.  Proceedings  of  the  Human  Factors  Society  -  29th  Annual 
Meeting. 

Human  factors  engineering  is  a  discipline  which  has  not  had  widespread 
application  in  commercial  product  development,  but  through  the  proposed  usa¬ 
bility  laboratory,  experts  may  incorporate  this  approach  into  their  product 
evaluation.  Describes  the  structure  of  a  usability  laboratory. 

Bettenhausen,  K. ,  &  Murnighan,  J.K.  (1985).  The  emergence  of  norms  in  com¬ 
petitive  decisionmaking  groups.  Administrative  Science  Quarterly,  30,  350-372. 

Interaction  among  group  members  in  nineteen  newly  formed  decision  making 
groups  (composed  of  five  persons  each)  was  studied  to  understand  how  social 
norms  form.  The  behavior  of  these  group  members  served  as  the  foundation  in 
constructing  a  model  of  norm  development  whereby  uncertainty  over  appropriate 
behavior  directs  people  to  utilize  their  past  experiences  in  similar  social 
settings  as  scripts.  By  using  these  experiences  they  determine  behaviors  in 
their  current  situations.  The  results  of  gaining  knowledge  about  how  norms 
develop  would  grant  the  capability  of  "managing"  this  limitation  in  group  deci¬ 
sion  making  situations. 

Chu,  Y. ,  &  Rouse,  V.B.  (1979,  December).  Adaptive  allocation  of  decisionmaking 
responsibility  between  human  and  computer  in  multitask  situations.  IEEE 
Transactions  on  Systems,  Han,  and  Cybernetics,  SMC-9(12),  769-778. 

Addresses  the  question  of  how  decisionmaking  responsibilities  should  be 
allocated  between  humans  and  increasingly  intelligent  computers  as  systems 
become  more  automated.  The  authors  maintain  that  with  "adaptive  allocation" 
the  allocation  of  responsibility  is  contingent  upon  the  state  of  the  system  and 
decisionmakers.  The  decisionmaker  most  capable  of  performing  a  task  at  a  par¬ 
ticular  time  will  be  subject  to  complete  it.  An  experimental  study  involving  a 
flight  management  task  illustrates  this  general  strategy. 

Cohen,  M.S.,  &  Freeling,  A.N.S.  (1981,  February).  The  impact  of  information  on 
decisions:  command  and  control  system  evaluation  (Technical  Report  81-1  ). 

Falls  Church,  Virginia:  Decision  Science  Consortium,  Inc. 

This  study  applies  decision  analytic  concepts  to  the  problem  of  determining 
value  of  information  in  the  design  and  evaluation  of  information  systems. 
Although  many  current  systems  furnish  an  abundance  of  data,  the  decision  maker 
is  often  unable  to  pick  out  the  data  essential  to  the  problem.  Necessary  in¬ 
formation  is  that  which  has  a  high  probability  of  changing  a  decision  and  im¬ 
proving  decision  outcome.  This  study  modifies  standard  VOI  techniques  in  order 
to  simplify  them  and  encourage  their  usage. 

Collins,  A.  (1978).  Fragments  of  a  theory  of  human  plausible  reasoning.  In 
D.L.  Waltz  (Ed.),  Theoretical  Issues  in  Natural  Language  Processing  -  2,  Urbana 
Champaign:  University  of  Illinois,  194-201. 

Discusses  a  theory  of  human  plausible  reasoning  which  purports  to  exemplify 
how  various  configurations  of  information  affect  the  surety  of  drawn  conclu¬ 
sions.  The  theory  is  computational  in  nature  and  was  constructed  from  an  ex¬ 
periment  in  which  an  analysis  was  made  of  the  answers  people  gave  to  routine 
questions. 
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Collins,  A.,  &  Stevens,  A.L.  (1981 ).  A  cognitive  theory  of  interactive  teach¬ 
ing.  To  appear  in  C.M.  Eeigeluth  (Ed.),  Instructional  Design  Theories  and 
Models;  An  Overview,  New  York:  Academic  Press. 

The  authors  propose  a  theory  of  inquiry  teaching  whereby  students  construct 
theories  by  dealing  with  specifications  and  subsequently  apply  these  theories 
to  new  cases.  The  authors  feel  that  as  a  result  of  this  teaching,  reasoning 
skills,  such  as  forming  and  testing  hypotheses,  making  predictions,  and  knowing 
which  questions  to  ask,  to  name  a  few,  are  learned  and  applied.  It  is  recom¬ 
mended  that  these  techniques  be  applied  within  other  fields  of  study. 

Einhoro,  H.J.,  Hogarth,  R.M. ,  &  Klempner,  E.  (1977).  Quality  of  group  judg¬ 
ment.  Psychological  Bulletin,  84,  158-172. 

Deals  with  the  accuracy  of  group  judgment  within  and  between  different 
populations  of  groups.  The  quality  of  group  judgment  is  defined.  Results  as 
well  as  constraints  affecting  judgmental  quality  of  various  strategies  for 
merging  opinions  under  a  diversity  of  circumstances  are  discussed.  A  statisti¬ 
cal  procedure  is  introduced  that  considers  which  baselines  are  suitable  in  the 
evaluation  of  the  quality  of  group  judgment. 

Fischer,  G.W.  (1979,  July).  Utility  models  for  multiple  objective  decisions; 
do  they  accurately  represent  human  preferences?  The  Journal  for  the  American 
Institute  for  Decision  Sciences,  10(3),  451-479- 

This  paper  is  written  for  those  decision  scientists  who  do  not  utilize 
Multivariate  Utility  (MAU)  modeling  procedures  because  they  doubt  that  these 
procedures  are  reliable  in  providing  a  true  representation  of  human  prefer¬ 
ences.  The  author's  goal  is  to  persuade  many  that  the  MAU  approach  has  many 
useful  applications.  Two  approaches  to  MAU  modeling  as  well  as  relevant  psy¬ 
chological  research  are  examined.  Covered  in  this  article  are  the  following 
topics:  assigning  utility  values  to  decision  outcomes,  holistic  utility  assess¬ 
ment  methods,  an  overview  of  MAU  theory,  methods  for  assessing  decomposed  MAU 
functions,  common  objections  to  MAU  models,  and  validating  MAU  models  and  as¬ 
sessment  procedures. 

Fischhoff,  B. ,  &  Johnson,  S.  (1985,  March).  The  possibility  of  distributed 
decision  making.  Prepared  for  Workshop  on  Political-Military  Decision  Making, 
The  Hoover  Institution,  Stanford  University. 

Reviews  existing  problems  of  decision  making  distribution  when  dealing  with 
several  individuals  or  groups.  The  authors  propose  solutions  for  complications 
which  may  arise  as  a  result  of  using  these  systems,  and  discuss  design  and 
management  of  distributed  decision  making.  Outlined  are  possible  limitations 
which  could  affect  an  organization. 
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Goldstein,  I.  (1978,  October).  Developing  a  computational  representation  for 
problem  solving  skills  (Artificial  Intelligence  Memo  495) •  Paper  to  appear 
in  Proceedings  of  the  Carnegie  Mellon  Conference  on  Problem  Solving  and  Educa¬ 
tion:  Issues  in  Teaching  and  Research. 

Presents  a  problem  solving  model  which  originated  from  the  study  of  com¬ 
puter  coaches  (advanced  computer  assisted  instruction).  This  model  depicts 
problem  solving  skills  as  an  evolving  set  of  rules  for  a  domain  influencing 
growing  representation  of  the  problem  and  administered  by  a  resource-limited 
problem  solver. 

Goldstein,  I.P.  (1979)*  The  genetic  graph:  a  representation  for  the  evolution 
of  procedural  knowledge.  International  Journal  of  Man-Machine  Studies,  11, 
51-77. 

A  model  of  the  evolution  of  rule-structured  knowledge  which  is  considered  a 
cornerstone  of  the  development  of  computer-based  coaches  is  illustrated.  A 
graph  structure  depicting  the  concepts  of  generalization,  correction,  and  re¬ 
finement  is  displayed  and  defined.  Various  genetic  graph  formulations  pertain¬ 
ing  to  reasoning  skills  used  in  playing  an  elementary  mathematical  game  are 
examined. 

Hart,  S.L.  (1985).  Towa  d  quality  criteria  for  collective  judgments. 
Organizational  Behavior  and  Human  Decision  Processes,  56,  209-228. 

Addresses  the  subject  of  group  decision  making  by  presenting  evaluative 
criteria,  which  were  derived  from  a  review  of  prevailing  literature,  for  judg¬ 
ing  when  a  "high-quality  outcome"  has  been  reached.  A  major  goal  which  is 
hoped  to  be  generated  by  this  project  is  the  construction  of  a  set  of  empiri¬ 
cally  derived  criteria  against  which  outcome  quality  may  be  judged.  The  most 
effective  procedures  for  the  achievement  of  a  high  quality  collective  judgment 
are  suggested  as  a  result  of  six  factual  group  problem-solving  cases  which 
were  evaluated  by  use  of  a  specially  developed  questionnaire. 

Hogarth,  R.M.  (1975).  Decision  time  as  a  function  of  task  complexity.  In 
Wendt/Velk  (Eds.).  Utility,  Probability,  and  Human  Decision  Making  (pp.  321- 
338).  Dordrecht-Holland:  D.  Riedel  Publishing  Company. 

Discusses  the  association  between  decision  time  and  task  complexity.  A 
mathematical  model  is  thoroughly  explained  which  examines  this  association. 

The  model  is  described  as  having  several  observable  variables  linked  to  an 
unobservable  variable  of  the  psychological  function,  i.e.,  cognitive  strain. 

It  is  suggested  that  when  a  decision  maker  is  faced  with  different  alterna¬ 
tives,  he  considers  two  kinds  of  cost:  the  time  involved  in  the  decision  pro¬ 
cedure,  and  the  cost  of  making  errors. 
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Hogarth,  R.M. ,  &  Einhorn,  H.J.  (1976,  January,  in  press).  Optimal  strategies 
for  personnel  selection  when  candidates  can  reject  offers.  Journal  of  Busi¬ 
ness. 

Discusses  personnel  decision  making  with  the  inclusion  of  the  possibility 
of  rejection  of  job  offers  by  candidates,  which  the  authors  recognize  as  an 
overlooked  factor  in  formal  models  of  decision  making.  Perfect  Expected  Accep¬ 
tance  Gain  and  Expected  Acceptance  Gain  are  promoted  and  demonstrate  signifi¬ 
cant  value  for  assessing  the  effects  of  various  selection  strategies.  An 
interactive  computer  program  is  described.  It  is  capable  of  being  utilized  for 
sensitivity  analysis  as  a  tool  in  the  evaluation  of  important  variations  in 
subjective  inputs,  which  affect  changes  in  decision  variables  and  their  possi¬ 
ble  economic  results. 

Jin,  V.Y.  (1985,  May).  Delays  for  distributed  decisionmaking  organizations. 
Unpublished  thesis,  Massachusetts  Institute  of  Technology,  Cambridge, 
Massachustetts . 

In  this  thesis  an  algorithm  for  computing  time  delay  in  the  operation  of 
distributed  decision  making  systems  (DDM)  is  produced.  Interactors  in  DDM 
organizations  are  investigated  and  asynchronous  protocols  that  enrich  perform¬ 
ance  are  constructed. 

Johnson,  P.E.  (1980,  June,  in  press).  Cognitive  models  of  medical  problem 
solvers.  In  D.  Connelly,  E.  Benson,  &  D.  Burke  (Eds.),  Clinical  Decision  Mak¬ 
ing  and  Laboratory  Use.  Minneapolis,  Minnesota:  University  of  Minnesota 
Press. 

In  this  discussion  of  cognitive  models  of  medical  problem  solvers  the 
principal  theme  is  that  similarities  outweigh  differences  when  comparing 
individual  problem  solvers.  A  diagnostic  reasoning  model  of  physician 
knowledge  is  proposed  which  depicts  what  patients  with  certain  diseases  or 
conditions  should  "look  like." 

Johnson- Laird,  P.N.  (1983).  Mental  models,  towards  a  cognitive  science  of 
language,  inference,  and  consciousness.  Cambridge,  Massachusetts:  Harvard 
University  Press. 

This  chapter  from  Johnson- Laird's  book  defines  "mental  models,"  explains 
how  they  are  constructed,  and  interprets  them.  The  theory  itself  is  designed 
to  explain  the  higher  processes  of  -jognition,  particularly  comprehension  and 
inference. 

Jungermann,  H.  ( 1980a).  Speculations  about  decision- theoretic  aids  for  per¬ 
sonal  decisionmaking.  Acta  Psychologica,  45.  7-34. 

Deal*  with  techniques  which  assist  people  in  making  personal  decisions. 
Personal  decisions  are  defined  as  those  where  the  person  making  the  decision 
must  f.  ce  tha  ramifications  of  his  actions.  Even  though  this  distinction  may 
seem  blurred  between  those  persons  making  decisions  for  themselves  versus  oth¬ 
ers,  the  Author  holds  the  opinion  that  not  enough  attention  in  applied  decision 
research  it*  given  to  personal  decisionmaking.  Structuring  the  problem,  as¬ 
sessing  O'  s  value  system,  and  examining  alternative  options  are  addressed  in 
helping  one  to  arrive  at  a  decision. 
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Jungermann,  H.  ( 1980b,  September).  Structural  modeling  of  decision  problems. 
Paper  presented  at  the  88th  Annual  Convention  of  the  American  Psychological 
Association  (APA),  Montreal. 

The  author's  interest  lies  with  the  first  stage  in  the  process  of  decision 
making,  in  which  the  problem  is  represented  ^translated  cognitively  into  a 
structure  that  represents  the  situation).  It  is  proposed  that  the  questions 
one  asks  influence  the  answers  one  gets,  i.e.,  various  methods  of  posing  a 
decision  problem  to  a  person  stimulates  him  to  "reference  structure"  different 
sets  of  knowledge.  Examples  are  given  of  the  phases  of  the  decision  making 
process,  the  structuring  of  knowledge,  and  fault  trees. 

Kassirer,  J.P. ,  &  Gorry,  G.A.  (1978).  Clinical  problem  solving:  a  behavioral 
analysis.  Annals  of  Internal  Medicine,  89.  245-255. 

Reports  on  some  of  the  specific  tactics  and  overall  strategies  composing 
the  problem-solving  behavior  of  trained  clinicians  as  they  were  "taking  the 
history  of  the  present  illness."  The  tape  recorded  behavior  of  this  inter¬ 
change  between  a  clinician  and  simulated  patient  was  analyzed.  The  authors 
suggest  that  the  information  has  potential  value  in  the  field  of  medical  educa¬ 
tion  and  in  devising  computer  programs  to  assume  the  process  of  diagnosis. 

Klein,  G.A.  (1980,  October).  Automated  aids  for  the  proficient  decision  maker. 
Proceedings  of  the  Smith  Conference  on  Cybernetics  and  Society. 

Proposes  a  comparison- guided  model  of  decision-making,  maintaining  that 
proficient  decision  makers  are  able  to  recognize  and  subsequently  place  new 
situations  parallel  to  ones  previously  experienced.  The  types  of  decisions 
discussed  are  complex,  those  with  unusual  conditions.  It  is  asserted  that  a 
redefinition  of  the  problem  by  the  expert  will  allow  utilization  of  all  re¬ 
sources  which  may  yield  a  solution  to  the  problem,  rather  than  using  a  decision 
aid  requiring  "rule- following."  The  author  believes  that  proficiency  is  not 
simply  following  rules  and  procedures. 

Morris,  N.M. ,  Rouse,  W.B.,  &  Frey,  P.R.  (1985,  February).  Adaptive  aiding 
for  symbiotic  human- computer  control:  conceptual  model  and  experimental 
approach  (Air  Force  Aerospace  Medical  Research  Laboratory  Technical  Report 
#84-072) .  Wright-Patterson  Air  Force  Base,  Ohio:  Search  Technology,  Inc. 

In  this  report,  development  of  an  experimental  approach  to  be  used  in  the 
investigation  of  the  advantages  of  adaptive  aiding  is  recapitulated.  Implica¬ 
tions  for  the  design  of  adaptive  aids  are  examined  and  a  conceptual  framework 
is  outlined.  The  task  environment  consists  of  two  competing  tasks  which  must 
be  performed  concurrently:  a  target  spotting  task  and  a  tracking  task.  Re¬ 
sults  indicate  that  the  manipulation  of  spotting  task  difficulty  affects 
performance. 

Morris,  P.A.  (1975,  February).  Modeling  experts.  Manuscript  submitted  for 
publication. 

Addresses  the  problem  of  expert  resolution.  A  model  is  presented  which 
allows  the  decision  maker  to  arrive  at  a  solution,  whether  it  is  from  a  single 
expert  or  a  group  of  experts.  The  term  "surrogate  prior"  is  defined  and  shown 
to  play  a  role  in  measuring  the  joint  information  contained  in  the  probability 
assessments  generated  by  a  panel  of  experts  within  the  case  of  the  multi- 


35 


expert.  This  modeling  approach  forms  the  basis  for  analytical  thinking  about 
the  interaction  between  experts  and  the  manner  in  which  probabilities  should  be 
derived  from  experts. 

Newman,  J.R.  (1977,  August).  Differential  weighting  for  prediction  and  deci¬ 
sion  making  studies:  a  study  of  ridge  regression  (SSRI  Research  Report 
77-1 ) . University  of  Southern  California: Social  Science  Research  Institute. 

This  study  describes  the  conditions  which  are  appropriate  for  differential 
or  simple  unit  weighting  of  predictor  variables  in  prediction  and/or  decision 
studies.  A  regression  model,  ridge  analysis  (RIDGE),  is  proposed  as  a  substi¬ 
tution  for  the  ordinary  least  squares  (OLS)  regression  analysis  because  of 
potential  difficulties  with  OLS  analysis.  The  RIDGE  method  of  estimation  may 
also  be  used  as  a  substitute  for  cross  validation. 

Norman,  D.A.  (1981b).  A  psychologist  views  human  processing;  human  errors  and 
other  phenomena  suggest  processing  mechanisms.  Paper  presented  at:  IJCAI, 
Vancouver,  British  Columbia. 

Discusses  human  processing  structures  by  arguing  for  a  different  kind  of 
processing  mechanism  than  that  which  is  most  commonly  referred  to  in  Artifi¬ 
cial  Intelligence.  The  form  of  human  error  is  of  major  importance  in  this 
study.  A  distinction  is  made  between  two  major  classes  of  errors,  mistakes 
versus  slips,  and  these  two  types  are  examined. 

Norman,  D.A.  ( 1981c,  August).  Steps  toward  a  cognitive  engineering:  system 
images,  system  friendliness,  mental  models.  Paper  presented  at  Symposium  on 
Models  of  Human  Performance,  ONR  Contractor's  meeting,  La  Jolla,  California. 

Contends  that  current  inappropriate  system  design  does  not  allow  optimum 
efficacy  between  the  users  and  machines.  Through  the  study  of  human  error,  it 
is  discovered  that  human  usage  should  be  a  factor  warranting  primary  considera¬ 
tion  by  designers.  The  author  proposes  a  cohesive  system,  a  "friendly  system", 
composed  of  the  principles  underlying  both  cognitive  science  and  human  factors. 

Pangaro,  P.A. ,  &  Nicoll,  J.F.  (1983,  February).  Deleting  the  knowledge  engi¬ 
neer:  the  practical  design  of  intelligent  support  software  based  on  Pask's 
proto-logic  Lp.  Paper  presented  at  the  Se-iinar  "The  Application  of  Machine  In¬ 
telligence  to  Defense  Systems",  Portland,  UK. 

Discusses  the  role  of  the  knowledge  engineer,  a  person  who  builds  an  expert 
system  in  conjunction  with  an  expert  in  the  subject  field.  Believes  that  the 
expert  should  communicate  directly  fith  the  computer,  thus  eliminating  the 
need  for  a  knowledge  engineer.  An  Intelligent  Support  System  (ISS)  is  pre¬ 
sented  as  an  ideal  solution  to  providing  sophisticated  support  in  expert  sys¬ 
tems  in  three  different  roles:  the  training  of  novices,  tactical  or  opera¬ 
tional  support,  and  strategic  planning.  The  philosopy  is  that  there  should  be 
only  a  narrow  gap  between  the  creators  and  operate  s  of  an  intelligent  support 
system. 
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Pearl,  J.f  Kim,  J.H.,  &  Fiske,  R.  (1981,  December).  An  experimental  evaluation 
of  the  effectiveness  of  two  decision  structuring  methods.  Prepared  for  Wayne 
Zachary  (Ed.),  Computer  Augmentation  of  Human  Decision  Making.  New  York: 

Gordon  and  Breech. 

An  experimental  evaluation  is  conducted  on  the  effectiveness  of  decision- 
tree  elicitation  and  goal-directed  structuring.  To  analyze  these  two  structur¬ 
ing  procedures,  a  computer  simulation  of  a  hypothetical  decision  making  envi¬ 
ronment,  Goal-Directed  Decision-Structuring  System  (GODDESS),  was  applied. 

This  system  promoted  an  objective  evaluation  of  each  decision  instituted  by  the 
subjects.  The  main  topics  of  discussion  are  the  simulated  model,  the  experi¬ 
mental  approach,  and  the  results.  The  results  illuminate  the  strengths  and 
weaknesses  of  the  two  decision  structuring  methods.  The  authors  contend  that 
the  supremacy  of  the  goal-directed  approach  would  be  recognized  in  a  situation 
where  the  difference  in  performance  between  long-range  and  short-range  planners 
is  more  strongly  emphasized. 

Phelps,  R.H. ,  Halpin,  S.M. ,  <S  Johnson,  E.M.  (1981,  January).  A  decision 
support  framework  for  decision  aid  designers  (aRI  Technical  Report  504). 
Alexandria,  Virginia:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

Describes  a  decision  support  framework  to  be  utilized  by  developers  of 
decision  aids  to  assist  them  in  the  organization  and  categorization  of  decision 
aiding  procedures.  This  framework  succinctly  distinguishes  between  two  catego¬ 
ries  of  aids  and  exemplifies  the  development  and  execution  of  the  aid  system. 
The  authors  suggest  that  utilization  of  this  framework  could  result  in  maximum 
productivity  of  their  systems.  Army  Intelligence  decision  making  examples  are 
cited  which  typify  the  components  of  the  framework.  This  decision  support 
framework  is  designed  to  represent  the  decision  support  components,  not  the 
decision  process  itself. 

Posa,  T.  (1985,  October).  Software  helps  busy  executives  analyze,  make  key 
decisions.  Infoworld,  2(41),  43-44. 

Decisionmap,  published  by  Softstyle  Inc.,  is  decision  support  software  (run 
on  the  Macintosh)  through  which  one  is  able  to  design  his/her  own  decision 
model.  After  choosing  criteria  on  which  the  decision  is  to  be  based,  factors 
are  weighed  according  to  importance  and  possible  solutions  are  ranked.  The 
author  advocates  its  usage,  emphasizing  the  ease  of  learning  and  minimal  amount 
of  time  involved. 

Rouse,  S.H. ,  Rouse,  W.B.,  &  Hammer,  J.M.  (1982,  August).  Design  and  evaluation 
of  an  onboard  computer-based  information  system  for  aircraft.  IEEE 
1 ransactions  on  Systems,  Man,  and  Cybernetics,  SMC-12(4) ,  451-463. 

This  report  concerns  computer-aided  human  information  seeking.  It  is  pro¬ 
posed  that  a  computer-based  information  system  can  considerably  reduce  the 
frequency  of  human  errors.  Various  types  of  information  and  styles  of  presen¬ 
tation  are  evaluated.  Results  are  submitted  which  support  the  hypothesis  that 
computer  aiding  is  often  more  desirable  than  computerizing  in  that  the  goal 
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should  be  for  computers  to  help  humans  overcome  limitations  and  utilize  their 
best  skills,  rather  than  replace  human  operators.  The  design  of  an  onboard 
computer-based  information  system  for  aircraft  is  explained. 

Rouse,  V.B.  (1983).  Models  of  human  problem  solving:  detection,  diagnosis, 
and  compensation  for  system  failures.  Automatica,  19(6),  613-625. 

Covers  models  of  human  problem  solving  with  an  emphasis  on  those  models 
which  predict  human  behavior  and  performance  to  support  design  and  evaluation. 
Areas  which  are  of  particular  concern  in  this  article  are  detection,  diagnosis, 
and  compensation  of  system  failures.  Proposed  is  an  outline  of  a  model  which 
may  influence  the  integration  and  progress  of  understanding  human  problem  solv¬ 
ing. 

Rouse,  W.B.  (1984).  Design  and  evaluation  of  computer-based  decision  support 
systems.  To  appear  in  S.J.  Andriole  (Ed.),  Microcomputer  Decision  Support 
Systems.  Wellesley,  MA:  QED  Information  Sciences. 

Argues  that  by  integrating  design  and  evaluation  in  computer-based  decision 
support  systems,  improvements  in  the  efficiency  and  effectiveness  of  the  proc¬ 
ess  will  be  the  result.  Because  systems  are  becoming  increasingly  complex, 
decision  suport  systems  which  aid  humans  in  decision  making  and  problem  solving 
are  essential. 

Rouse,  W.B.,  &  Rouse,  S.H.  (1984).  Human  information  seeking  and  design  of 
information  systems.  Information  Processing  and  Management,  20 ( 1 ) ,  129-138. 

Organizes  current  literature  from  various  disciplines  (i.e.,  psychology, 
management,  systems  engineering,  computer  science,  and  library  science)  to 
develop  the  topic  of  human  information  seeking  and  information  systems.  The 
nature  of  information  seeking  is  defined,  and  through  discussion,  the  contrast 
is  made  between  information  seeking  and  information  processing.  The  difficulty 
in  defining  and  measuring  the  value  of  information  affects  the  progress  which 
may  be  made  in  developing  frameworks  and  formulating  methods  to  evaluate  this 
area  of  concern.  Human  information  seeking  is  shown  to  be  a  integral  part  of 
decision  making  and  problem  solving. 

Saleh,  J. ,  Leal,  A.,  Kim,  J.,  &  Pearl,  J.  (1979,  October).  Progress  toward  a 
goal-directed  decision  support  sytem  (Technical  Report  UCLA-ENG-CSL-7973 ) • 
University  of  California,  Los  Angeles:  School  of  Engineering  and  Applied 
Science. 

A  computerized  decision  support  system  is  described  which  was  developed  by 
the  authors  as  an  effective  replacement  for  decision-tree  representation.  They 
state  that  the  approach  is  preferable  because  of  the  clarity  and  purposefulness 
it  provides.  This  goal-directed  method  focuses  on  a  single  objective  at  any 
given  time  and  instructs  the  user  to  ignore  all  side  effects.  An  extensive 
amount  of  graphs,  structures,  and  representations  are  provided. 
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Schweiger,  D.M. ,  Anderson,  C.R.,  &  Locke,  E.A.  (1985).  Complex  decision  mak¬ 
ing:  a  longitudinal  study  of  process  and  performance.  Organizational  Behav¬ 
ior  and  Human  Decision  Processes,  36,  245-272. 

Simultaneous  verbal  protocols  were  employed  to  study  individual  decision 
making  processes.  The  authors  describe  the  protocol  analysis  procedure  used 
and  present  the  coding  scheme  developed  to  analyze  the  data.  Results  of  two 
experiments  show  that  thought  processes  used  in  complex  decision  making  are 
associated  with  subsequent  performance:  subjects  using  causal  analysis  per¬ 
formed  more  successfully  than  those  who  did  not. 

Shanteau,  J.  (1975).  Averaging  versus  multiplying  combination  rules  of  infer¬ 
ence  judgment.  Acta  Psychologica,  39,  83-89. 

Two  combination  rules,  averaging  and  multiplying,  which  describe  sequential 
inference  judgments,  were  compared  through  three  experiments  in  this  study. 

Both  of  these  rules  depict  how  successive  pieces  of  evidence  are  combined  with 
previous  judgments  to  form  new  judgments.  Usage  of  the  averaging  rule  was  sup¬ 
ported  but  the  descriptive  ability  of  the  Bayesian  approach  (and  other 
approaches  which  use  the  multiplying  rule)  was  questioned.  It  may  be  inferred 
from  the  averaging  result  that  judgments  may  incorporate  processing  rules  dif¬ 
ferently  than  previously  speculated. 

Shanteau,  J. ,  &  O'Reilly,  M.  (1981,  November).  Beyond  linearity  in  decision 
making:  the  impact  of  3ynergisms  with  representative  designs.  Paper  presented 
at  the  Psychonomic  Society  meeting,  Philadelphia,  Pennsylvania. 

Evaluated  are  the  effects  of  synergisms  on  nonorthogonal  designs.  It  is 
determined  that  multiplicative  synergisms  have  an  influence  on  nonorthogonal 
designs.  Methodology  currently  in  use  in  the  analysis  of  these  designs  is 
questioned.  Incongruity  is  found  in  the  stimulus  cue  sets;  some  reveal  the 
presence  of  synergisms  better  than  others. 

Smillie,  R.J.,  &  Ayoub,  M.A.  ( 1 980) .  Job  performance  aids:  evaluation  of 
design  alternatives  via  network  simulation.  Ergonomics,  23(4) ,  319-339. 

Discusses  the  utilization  of  job  performance  aids  (JPAs),  which  are  defined 
as  plans  that  allow  the  human  component  of  a  system  to  perform  some  function  he 
could  not  otherwise  accomplish  without  extensive  training  or  complex  informa¬ 
tion  processing.  Through  the  employment  of  computer  simulation  (as  opposed  to 
laboratory  experimentation)  different  combinations  of  job  performance  aid  for¬ 
mats  and  the  effect  of  stress  on  task  performance  with  a  JPA  is  analyzed.  The 
use  of  computer  simulation  allows  one  to  estimate  and  evaluate  JPA  combinations 
under  varying  conditions. 

Smith,  S.L.,  &  Mosier,  J.N.  (1984,  September).  Design  guidelines  for  user- 
system  interface  software  (Technical  Paper  ESD-TR-84-190).  Bedford,  ” 

Massachusetts:  The  Mitre  Corporation. 

Provides  guidelines  for  the  design  of  user  interface  software  to  aid  de¬ 
signers,  system  analysts,  teachers,  students,  human  factors  practitioners,  and 
researchers  in  making  this  process  more  efficient.  The  focus  is  on  six  func¬ 
tional  areas:  data  entry,  data  display,  sequence  control,  user  guidance,  data 
transmission,  and  data  protection. 
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Stevens,  A.,  Collins,  A.,  &  Goldin,  S.E.  (1979).  Misconceptions  in  student's 
understanding.  Int.  J.  Man-Machine  Studies,  1 1  ,  145-156. 

Proposed  in  this  research  report  is  a  system  which  tutors  students  with 
intelligent  computer-aided  instruction  (ICAI)  by  diagnosing  and  rectifying 
conceptual  misunderstandings.  A  theory  based  on  examination  of  tutoring 
dialogs  is  the  basis  for  this  system.  The  goal  structure  of  a  tutor  and  types 
of  conceptual  bugs  are  specified. 

Waldrop,  G.P. ,  &  Lane,  N.E.  (1985).  Computer-based  instruction  (CBl):  toward 
a  user-oriented  technology  data  base.  Proceedings  of  the  Human  Factors 
Society  -  29th  Annual  Meeting. 

The  focus  of  this  paper  is  on  two  issues  originating  from  computer-based 
instruction  (CBl)  research  and  development  analysis  in  support  of  the  CBI  data 
base  design.  The  two  points  of  concern  deal  with:  (a)  the  importance  of  spe¬ 
cific  attention  to  the  characteristics  and  requirements  of  potential  users  (of 
CBI  and  of  the  data  base),  and  (b)  the  necessity  of  having  a  we  11- formula  ted 
classification  schema  for  description  of  the  features  of  CBI  systems  in  the 
form  of  data  storage  and  retrieval  cues.  The  goal  of  this  technology  analysis 
and  synthesis  is  to  describe  current  and  near  future  capabilities  of  CBI  in  the 
Department  of  Defense  (DOD)  environment. 

Weisbrod,  R.L. ,  Davis,  K.B.,  &  Freedy,  A.  (1975).  Adaptive  utility  assessment 
in  dynamic  decision  processes:  an  experimental  evaluation  of  decision  aiding. 
Proceedings  of  Conference  on  Systems,  Man,  and  Cybernetics  Society,  San 
Francisco,  California. 

Supports  usage  of  an  Adaptive  Dynamic  Decision  Aiding  Mechanism  (ADDAM) 
system  to  aid  decision  makers  in  their  administration  of  dynamic  decision 
tasks.  The  effectiveness  of  the  decision  aiding  system  was  examined  through  an 
experimental  study  which  used  a  realistic  decision  task.  ADDAM  allows  the 
operator  to  organize  his  own  in-context  behavior  into  a  methodical  mathematical 
framework.  The  purpose  of  this  strategy  is  not  simply  to  present  a  model  for 
decision  making  behavior,  but  also  to  furnish  a  foundation  for  decision  aiding. 

Zachary,  W. ,  &  Hopson,  J.  (1981  a).  A  methodology  for  decision  augmentation 
system  design  (Report  No.  81-2201).  American  Institute  of  Aeronautics  and 
Astronautics,  Inc. 

Proposes  steps  and  guidelines  which  will  enable  airborne  decision  makers  to 
enhance  their  performance.  This  methodological  approach  is  intended  to  allow 
designers  to  effectively  convert  system  performance  objectives  into  full  system 
designs.  Cites  some  difficulties  with  decision  augmentation  systems  and  out¬ 
lines  steps  to  design  and  organize  these  systems. 

Zachary,  W. ,  &  Hopson,  J.  (1981b,  June).  Scope  and  requirements  of  a  framework 
for  decision  augmentation  system  design.  Willow  Grove,  Pennsylvania: 

Analytics  Inc. 

Provides  support  for  the  notion  that  the  implementation  of  Decision  Aug¬ 
mentation  Systems  (DASs)  is  an  additional  requirement  for  properly  assessing 
and  developing  situations  in  order  to  make  optimal  decioions.  The  introduction 
of  computer-based  systems  which  support  managerial  decision  making  (Decision 
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Support  Systems,  DSSs)  are  not  sufficient  in  augmenting  human  decision  making 
performance  to  its  fullest  degree.  The  overall  structure  of  a  functional  and 
computational  DAS  is  presented  in  six  stages. 


User/Computer  Interface 

Alter,  S.  (1977,  February).  Why  is  man-computer  interaction  important  for 
decision  support  systems?  Interfaces,  7/2),  109-115. 

Argues  that  interactive,  conversational  decision  support  systems  do  not 
represent  their  true  identities  in  many  cases.  The  author  believes  that  better 
man-computer  interfaces  are  needed,  but  not  necessarily  by  managers  themselves. 
Suggests  that  the  focus  should  be  on  experts  who  understand  the  details  under¬ 
lying  a  support  system;  if  managers  do  not  learn  the  correct  usage  of  decision 
support  systems,  then  usage  should  be  discouraged  without  the  aid  of  an  inter¬ 
mediary. 

Andriole,  S.J.  (1982,  July/ August) .  The  design  of  microcomputer- based  personal 
decision  aiding  systems.  IEEE  Transactions  on  Systems,  Man,  and  Cybernetics, 
SMC-1 2(4) ,  463-469. 

Presents  an  approach  to  the  design  and  development  of  personal  computer- 
based  decision  aiding  systems  which  views  the  human  problem-solver  as  an  active 
user  who  must  be  able  to  use  the  computer  without  the  aid  of  a  problem-solving 
intermediary.  In  order  to  successfully  utilize  a  system,  the  author  states 
that  the  design  of  the  system  should  encompass  the  measurement  and  integration 
of  cognitive,  situational,  and  physiological  variables  which  should  inform  and 
alter  the  decision-aiding  process.  It  is  suggested  that  without  explicit  user 
requirements,  interactive  systems  fulfill  the  needs  of  the  designer  and  not  the 
target  user.  Problem  solvers  are  categorized  according  to  their  experience 
with  computer-based  systems  and  grouped  according  to  their  "cognitive  styles" 
in  order  to  formulate  computer-generated  information  and  strengthen  man's  in¬ 
teraction  with  the  computer.  The  user's  physiological  state  and  situational 
characteristics  are  examined  as  well. 

Baldwin,  J.T.,  &  Siklossy,  L.  (1977).  An  unobtrusive  computer  monitor  for 
multi-step  problem  solving.  Int.  J.  Han-Machine  Studies,  %  349-362. 

A  method  is  presented  for  designing  computer-assisted  instructional  (CAl) 
systems,  Unobtrusive  Problem  Solving  Monitors  (UPSM's).  These  systems  are 
aimed  at  monitoring  students  solving  multi-step  problems  and  provide  valuable 
guidance  to  students  based  on  prior  students'  input  history.  Students  may 
solve  problems  in  an  unguided  fashion  with  easily  accessible  help  if  needed.  A 
typical  example  of  a  UPSM  system  that  has  been  employed  in  plane  geometry  in¬ 
struction  is  summarized. 

Benbasat,  I.  &  Taylor,  R.N.  (1982,  July/ August) .  Behavioral  aspects  of  infor¬ 
mation  processing  for  the  design  of  management  information  systems.  IEEE 
Transactions  on  Systems,  Man,  and  Cybernetics,  SMC-1 2(4) ,  439-450. 

Cites  various  models  of  information  processing  which  account  for  human 
thinking  and  attempts  to  cover  the  capabilities,  as  well  as  the  inadequacies, 
in  information  processing  present  in  the  literature.  The  authors  believe  that 
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management  information  systems  (MIS)  can  be  radically  improved  by  understanding 
the  behavioral  processes  by  which  humans  process  information  and  make  choices. 
This  will  aid  in  eliminating  the  problem  of  "overloading"  the  human  user. 

Benel,  D.C.R.,  &  Pain,  R.F.  (1985).  The  human  factors  usability  laboratory  in 
product  evaluation.  Proceedings  of  the  Human  Factors  Society  -  29th  Annual 
Meeting. 

Human  factors  engineering  is  a  discipline  which  has  not  had  widespread 
application  in  commercial  product  development,  but  through  the  proposed  usa¬ 
bility  laboratory,  experts  may  incorporate  this  approach  into  their  product 
evaluation.  Describes  the  structure  of  a  usability  laboratory. 

Brooke,  J.B.,  Duncan,  K.D.,  &  Marshall,  E.C.  (1978).  Interactive  instruction 
in  solving  fault  finding  problems.  Int.  J.  Man-Machine  Studies,  10,  603-611. 

This  study  reviews  a  training  program  which  furnishes  supplementary  infor¬ 
mation  pertaining  to  the  relationship  between  remaining  faults  and  the  real¬ 
izable  indicators  during  fault  diagnosis.  An  interactive  computer  system 
provides  problem  solving  process  information  to  trainees  as  they  learn  to  diag¬ 
nose  faults.  A  flow  chart  is  given  which  displays  the  procedure  followed  by 
computer  programmers  in  controlling  a  fault  diagnosis  problem. 

Burke,  J.T.  (1986,  March).  The  next  step:  managing  battle  with  technology. 
ARMY,  56-60. 

Discusses  the  concept  <f  the  Battlefield  Management  System  (BMS)  and  advo¬ 
cates  its  role  in  improving  the  speed  and  effectiveness  of  combat  operations 
and  reducing  battlefield  stress.  Col.  (U.S.  Army  retired)  Burke  does  not  sug¬ 
gest  replacing  the  human  factor,  but  rather  applying  advanced  technology  to 
assist  soldiers,  leaders,  and  staff  people  in  coping  with  the  hectic  movement 
of  combat  action.  BMS  is  designed  with  the  purpose  of  allowing  humans  to  con¬ 
centrate  on  creative  decision  making  while  it  stores  and  processes  data  with 
great  speed.  BMS  would  be  an  immense  asset  to  the  battalion  S2  and  S3.  It  is 
argued  that  BMS  can  be  used  to  train  combat  crews  and  entire  units  through 
battlefield  simulation  and  will  save  millions  of  dollars,  a  substantial  amount 
of  time,  and  increase  efficiency  in  field  training. 

Burton,  R  R.,  &  Brown,  J.S.  ( 1 9T9 ) •  An  investigation  of  computer  coaching  for 
informal  learning  activities.  International  Journal  of  Man-Machine  Studies, 

11,  5-24. 

Addresses  the  problem  of  computer-based  coaching  by  presenting  a  coaching 
system  named  WEST,  structured  upon  the  game  "How  the  West  was  Won."  Analyzes 
this  system  in  order  to  discover  possible  limitations  and  to  yield  tutorial 
strategies  which  should  be  an  integral  part  of  a  successful  coaching  system. 

As  a  result  of  the  increase  in  use  of  personal  computers,  computer-based  games 
are  widely  played  and  may  provide  a  beneficial  environment  for  learning  which 
is  informal.  A  limitation  is  that  a  certain  amount  of  tutoring  is  necessary  to 
facilitate  proper  learning. 
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Caruso,  D.E.  (1969,  April).  Tutorial  programs  for  operation  of  on-line  re¬ 
trieval  systems.  Paper  presented  before  the  Division  of  Chemical  Literature, 
157th  Meeting,  ACS,  Minneapolis,  Minnesota. 

Discusses  the  need  for  an  increase  of  Computer-Aided  Instruction  (CAI) 
programs  in  on-line  computer  systems.  Interactive  on-line  computer  programs 
are  shown  to  allow  the  user  use  of  computer  services,  and  facilitate  meaningful 
contact  with  the  computer.  Three  examples  of  this  type  of  system  are  given 
which  were  developed  by  the  author  and  have  varying  user- training  characteris¬ 
tics.  The  author  purports  to  show  that  self-teaching  systems  can  create  a 
skilled  user  population. 

Chu,  Y. ,  &  Rouse,  W.B.  (1979,  December).  Adaptive  allocation  of  decisionmaking 
responsibility  between  human  and  computer  in  multitask  situations.  IEEE 
Transactions  on  Systems,  Man,  and  Cybernetics,  SMC-9 ( 1 2 ) ,  769-778. 

Addresses  the  question  of  how  decisionmaking  responsibilities  should  be 
allocated  between  humans  and  increasingly  intelligent  computers  as  systems 
become  more  automated.  The  authors  maintain  that  with  "adaptive  allocation" 
the  allocation  of  responsibility  is  contingent  upon  the  state  of  the  system  and 
decisionmakers.  The  decisionmaker  most  capable  of  performing  a  task  at  a  par¬ 
ticular  time  will  be  subject  to  complete  it.  An  experimental  study  involving  a 
flight  management  task  illustrates  this  general  strategy. 

Collins,  A.,  &  Adams,  M. J.  (1977).  Comparison  of  two  teaching  strategies  in 
computer  assisted  instruction.  Contemporary  Educational  Psychology,  2, 

133-148. 

In  three  experiments  comparing  a  method  of  teaching  developed  from  analysis 
of  human  tutors  (Tutorial  Mode)  versus  a  method  developed  from  programmed  in¬ 
struction  (Block-Test  Mode)  a  SCHOLAR  Computer  Assisted  Instruction  (CAl)  is 
employed.  The  goal  was  to  determine  which  facets  of  teaching  strategy  affected 
student  learning.  The  results  yield  the  conclusion  that  the  major  factor  af¬ 
fecting  student  learning  is  tutorial  strategy.  Because  CAI  has  the  capacity  to 
provide  a  mode  analoguous  to  human  tutors,  it  is  said  to  be  beneficial  in  the 
evaluation  and  implementation  of  different  teaching  strategies. 

Collins,  A.,  &  Stevens,  A.L.  ( 1 981 ) .  A  cognitive  theory  of  interactive  teach¬ 
ing.  To  appear  in  C.M.  Reigeluth  (Ed.),  Instructional  Design  Theories  and 
Models:  An  Overview,  New  York:  Academic  Press. 

The  authors  propose  a  theory  of  inquiry  teaching  whereby  students  construct 
theories  by  dealing  with  specifications  and  subsequently  apply  these  theories 
to  new  cases.  The  authors  feel  that  as  a  result  of  this  teaching,  reasoning 
skills,  such  as  forming  and  testing  hypotheses,  making  predictions,  and  knowing 
which  questions  to  ask,  to  name  a  few,  are  learned  and  applied.  It  is  recom- 
mendeJ  that  these  techniques  be  applied  within  other  fields  of  study. 
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Eklundh,  K.S.,  Marmolin,  H. ,  &  Hedin,  C.  (1985).  Experimental  evaluation  of 
dialogue  types  for  data  entry.  International  Journal  of  Man-Machine  Studies, 
22,  651-661. 

A  definite  interaction  between  type  of  dialogue  and  type  of  data  is  evident 
in  an  experiment  which  evaluated  four  fundamental  types  of  data  entry  dialogue 
concerning  speed.  It  was  found  that  subjects  in  this  experiment  preferred  an 
interaction  mode  which  maximized  speed  at  the  cost  of  less  feedback  and  less 
opportunity  for  error  control  in  the  command  dialogues.  There  were  no  signifi¬ 
cant  differences  between  the  number  of  errors  within  each  type  of  dialogue. 
Authors  contend  that  an  adaptable  man-computer  dialogue  is  necessary  with  an 
irregular  data  structure  in  performing  data  entry. 

Farquhar,  P.H.  (1982).  Utility  assessment  methods  (Working  Paper  81-5). 

Davis,  California:  University  of  California. 

Attempts  to  integrate  current  methods  for  evaluating  single-attribute  util¬ 
ity  functions.  Presents  various  new  methods  of  evaluation  which  may  be  benefi¬ 
cial  in  furthering  the  research  in  this  area,  emphasizing  the  varied  steps  in 
the  utility  assessment  process,  and  contrasting  methods  of  comparative  risk. 
Reviews  approximately  twenty-four  utility  assessment  methods. 

Goldstein,  I.  (1978,  October).  Developing  a  computational  representation  for 
problem  solving  skills  (Artificial  Intelligence  Memo  495).  Paper  to  appear 
in  Proceedings  of  the  Carnegie  Mellon  Conference  on  Problem  Solving  and  Educa¬ 
tion:  Issues  in  Teaching  and  Research. 

Presents  a  problem  solving  model  which  originated  from  the  study  of  com¬ 
puter  coaches  (advanced  computer  assisted  instruction).  This  model  depicts 
problem  solving  skills  as  an  evolving  set  of  rules  for  a  domain  influencing 
growing  representation  of  the  problem  and  administered  by  a  resource-limited 
problem  solver. 

Goldstein,  I.P.  (1979).  The  genetic  graph:  a  representation  for  the  evolution 
of  procedural  knowledge.  International  Journal  of  Man-Machine  Studies,  1 1 , 
51-77. 

A  model  of  the  evolution  of  rule-structured  knowledge  which  is  considered  a 
cornerstone  of  the  development  of  computer-based  coaches  is  illustrated.  A 
graph  structure  depicting  the  concepts  of  generalization,  correction,  and  re¬ 
finement  is  displayed  and  defined.  Various  genetic  graph  formulations  pertain¬ 
ing  to  reasoning  skills  used  in  playing  an  elementary  mathematical  game  are 
examined. 

Halpin,  S.M.,  Johnson,  E.M.,  <5  Thornberry,  J.A.  (1975,  August).  Cognitive 
reliability  in  manned  systems.  IEEE  Transactions  on  Reliability,  R-22(5) , 
165-170.  . . . . .  .  . 

Analyzes  cognitive  reliability  in  manned  systems,  which  the  authors  state 
is  an  intricate  function  of  attitudinal  and  structural  factors,  along  with 
their  interaction.  Cites  the  types  of  human  errors  which  may  transpire,  and 
discusses  factors  which  influence  the  chance  of  these  errors.  Due  to  the 
changing  role  of  man  in  complex  systems,  a  broadening  of  prevalent  approaches 
to  human  performance  reliability  must  be  sought.  In  addition  to  describing  the 
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tasks  which  are  becoming  progressively  more  important  in  man- machine  systems, 
the  term  "cognitive  reliability"  is  defined,  methods  of  classifying  cognitive 
errors  are  identified,  and  an  example  is  given  which  exemplifies  the  factors 
likely  to  affect  cognitive  errors. 

Hogarth,  R.M. ,  &  Einhorn,  H.J.  (1976,  January,  in  press).  Optimal  strategies 
for  personnel  selection  when  candidates  can  reject  offers.  Journal  of  Busi¬ 
ness. 

Discusses  personnel  decision  making  with  the  inclusion  of  the  possibility 
of  rejection  of  job  offers  by  candidates,  which  the  authors  recognize  as  an 
overlooked  factor  in  formal  models  of  decision  making.  Perfect  Expected  Accep¬ 
tance  Gain  and  Expected  Acceptance  Gain  are  promoted  and  demonstrate  signifi¬ 
cant  value  for  assessing  the  effects  of  various  selection  strategies.  An 
interactive  computer  program  is  described.  It  is  capable  of  being  utilized  for 
sensitivity  analysis  as  a  tool  in  the  evaluation  of  important  variations  in 
subjective  inputs,  which  affect  changes  in  decision  variables  and  their  possi¬ 
ble  economic  results. 

Huber,  G.P.  (1981,  November).  Cognitive  style  as  a  basis  for  designing 
MIS  and  DSS:  much  ado  about  nothing?  (Wisconsin  Working  Paper  11-81 -29 ) . 
Wisconsin:  University  of  Wisconsin-Madison. 

Argues  that  the  current  literature  on  cognitive  styles  is  inadequate  for 
determining  operational  Management  Information  Systems  and  Decision  Support 
Systems  (DSS)  guidelines.  He  further  states  that  cognitive  style  research  is 
unlikely  to  produce  valuable  guidelines. 

MacGregor,  D. ,  &  Slovic,  P.  (1985,  March).  Perceived  acceptability  of  risk 
analysis  as  a  decision-making  approach.  Eugene,  Oregon:  Decision  Research,  A 
Branch  of  Perceptronics. 

Investigates  a  methodology  for  understanding  how  people  evaluate  decision 
making  approaches  involving  health  and  safety  risk  factors.  Three  methods  for 
making  a  consumer  product  safety  decision  were  appraised  on  scales  relating  to 
their  perceived  acceptability,  logical  soundness,  completeness,  and  sensitivity 
to  moral  and  ethical  concerns.  An  effect  on  judgments  was  not  shown  from  hav¬ 
ing  knowledge  of  the  consequences.  An  implication  made  is  that  people's  accep¬ 
tance  of  risk  may  be  greatly  determined  by  the  manner  in  which  decision  risks 
are  made. 

Morehead,  D.R.  &  Rouse,  W.B.  (1985).  Human- computer  interaction  in  information 
seeking  tasks.  Information  Processing  and  Management,  19(4) ,  245-253* 

In  this  report,  the  authors  utilize  a  simplified  information  seeking  envi¬ 
ronment,  DBASE  (Data  Base  Access  and  Search  Environment),  in  an  experiment  to 
study  human  machine  interaction.  Included  in  this  topic  are  computer  aiding, 
availability  of  citation  lists,  data  base  structure,  and  type  of  search  tasks. 
It  is  indicated  from  the  results  of  this  experiment  that  data  base  structure 
and  type  of  search  task  may  be  combined  to  create  diversified  search  environ¬ 
ments,  which  greatly  influence  the  user's  search  performance. 
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Morris,  N.M.,  Rouse,  W.B.,  &  Frey,  P.R.  (1985,  February).  Adaptive  aiding 
for  symbiotic  human- computer  control;  conceptual  model  and  experimental 
approach  (Air  Force  Aerospace  Medical  Research  Laboratory  Technical  Report 
#84-072) .  Vright-Patterson  Air  Force  Base,  Ohio:  Search  Technology,  Inc. 

In  this  report,  development  of  an  experimental  approach  to  be  used  in  the 
investigation  of  the  advantages  of  adaptive  aiding  is  recapitulated.  Implica¬ 
tions  for  the  design  of  adaptive  aids  are  examined  and  a  conceptual  framework 
is  outlined.  The  task  environment  consists  of  two  competing  tasks  which  must 
be  performed  concurrently:  a  target  spotting  task  and  a  tracking  task.  Re¬ 
sults  indicate  that  the  manipulation  of  spotting  task  difficulty  affects 
performance. 

Newsted,  P.R.,  &  Wynne,  B.E.  (1976).  Augmenting  man's  judgment  with  inter¬ 
active  computer  systems.  Int.  J.  Man-Machine  Studies,  8,  29-59* 

Presents  a  computer-based  technique.  An  Interactive  Decision  System  (AIDS), 
with  the  goal  of  assisting  typical  managers  in  making  strategic  decisions  deal¬ 
ing  with  qualitative  issues  by  allowing  them  optimal  usage  of  their  available 
information  and  resources.  This  system  is  based  on  Kepner  and  Tregoe's  infor¬ 
mation  processing  techniques  utilized  in  their  decision  training  agendas.  A 
thorough  illustration  of  the  nine  steps  in  AIDS  and  a  discussion  of  the  program 
and  its  effectiveness  describe  this  approach. 

Norman,  D.A.  (1980,  August).  Errors  in  human  performance  (Report  No.  8004). 

La  Jolla,  California:  Center  for  Human  Information  Processing. 

This  report's  area  of  concentration  lies  in  human  error,  particularly  error 
made  by  skilled  operators  in  complex,  demanding  systems.  Through  this  analysis 
of  human  error,  various  applications  of  human  information  processing  are  de¬ 
lineated  and  a  classification  scheme  for  errors  is  outlined.  Short-term  memory 
and  human  attentional  limitations  are  examined  in  the  design  of  human-machine 
interfaces. 

Norman,  D.A.  (1981c,  August).  Steps  toward  a  cognitive  engineering:  system 
images,  system  friendliness,  mental  models.  Paper  presented  at  Symposium  on 
Models  of  Human  Performance,  ONR  Contractor's  meeting,  La  Jolla,  California. 

Contends  that  current  inappropriate  system  design  does  not  allow  optimum 
efficacy  between  the  users  and  machines.  Through  the  study  of  human  error,  it 
is  discovered  that  human  usage  should  be  a  factor  warranting  primary  considera¬ 
tion  by  designers.  The  author  proposes  a  cohesive  system,  a  "friendly  system", 
composed  of  the  principles  underlying  both  cognitive  science  and  human  factors. 

O'Shea,  T.  (1978).  A  self-improving  quadratic  tutor.  Int.  J.  Man-Machine 
Studies,  1 1 ,  97-124. 

The  lack  of  learning  capability  in  Computer  Assisted  Instruction  (CAl) 
programs  is  the  issue  addressed  in  this  research  report.  The  author's  goal  is 
to  present  an  alternative  teaching  strategy  for  instructing  students  on  ways  to 
solve  quadratic  equations.  Introduced  is  a  self- improving  quadratic  tutor 
which  teaches  the  solution  to  quadratic  equations  through  the  discovery  method. 
The  two  major  components  of  the  tutor  are  explained.  One  of  the  components  is 
an  adaptive  teaching  program  where  the  teaching  strategy  is  described  as  a  set 
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of  production  rules,  and  the  second  component  executes  the  self- improving  func¬ 
tion  of  the  system  by  making  experimental  changes  to  the  set  of  production 
rules. 

Rouse,  S.H.,  Rouse,  W.B.,  &  Hammer,  J.M.  (1982,  August).  Design  and  evaluation 
of  an  onboard  computer-based  information  system  for  aircraft.  IEEE 
Transactions  on  Systems,  Man,  and  Cybernetics,  SMC-12(4),  451-463. 

This  report  concerns  computer-aided  human  information  seeking.  It  is  pro¬ 
posed  that  a  computer-based  information  system  can  considerably  reduce  the 
frequency  of  human  errors.  Various  types  of  information  and  styles  of  presen¬ 
tation  are  evaluated.  Results  are  submitted  which  support  the  hypothesis  that 
computer  aiding  is  often  more  desirable  than  computerizing  in  that  the  goal 
should  be  for  computers  to  help  humans  overcome  limitations  and  utilize  their 
best  skills,  rather  than  replace  human  operators.  The  design  of  an  onboard 
computer-based  information  system  for  aircraft  is  explained. 

Rouse,  W.B.  (1984).  Design  and  evaluation  of  computer-based  decision  support 
systems .  To  appear  in  S.J.  Andriole  (Ed.),  Microcomputer  Decision  Support 
Systems.  Wellesley,  MA:  QED  Information  Sciences. 

Argues  that  by  integrating  design  and  evaluation  in  computer-based  decision 
support  systems,  improvements  in  the  efficiency  and  effectiveness  of  the  proc¬ 
ess  will  be  the  result.  Because  systems  are  becoming  increasingly  complex, 
decision  suport  systems  which  aid  humans  in  decision  making  and  problem  solving 
are  essential. 

Rouse,  W.B. ,  Rouse,  S.H.,  Hunt,  R.M.,  Johnson,  W.B. ,  &  Pelligrino,  S.J.  (1980, 
January).  Human  decision-making  in  computer-aided  fault  diagnosis  (Technical 
Report  434).  University  of  Illinois:  Coordinated  Science  Laboratory. 

Six  experiments  evaluating  the  effects  of  computer  aiding  on  performance 
are  described.  The  practicality  of  utilizing  context-free  computer-based 
simulations  to  instruct  troubleshooting  skills  is  also  examined.  A  goal  of 
this  research  study  is  to  increase  understanding  of  human  fault  diagnosis 
abilities.  Computer  aiding  significantly  lessened  the  number  of  tests  required 
to  diagnose  simple  problems  and  improved  consequent  unaided  performance. 

Slovic,  P.  (1971,  April).  Information  processing,  situation  specificity,  and 
the  generality  of  risk-taking  behavior.  ORI  Research  Bulletin,  1 1 (3) ,  1-22. 

Inter-task  consistency  of  individual  differences  was  ascertained  in  two 
structurally  comparable  risk-taking  tasks.  A  relationship  was  shown  between 
information  processing  and  situational  specificity.  As  a  result  of  the  experi¬ 
ment,  it  was  determined  that  the  two  response  modes  triggered  different  methods 
of  processing  information  pertaining  to  probabilities  and  payoffs  in  a  way  that 
influenced  individual  differences  and  reduced  inter- task  consistencies.  The 
implication  is  that  it  is  improbable  to  find  significant  correlations  between 
risk-taking  measures  in  structurally  different  settings  or  between  risk-taking 
and  different  behaviors. 
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Slovic,  P. ,  Fischhoff,  B. ,  &  Lichtenstein,  S.  (1977).  Behavioral  decision 
theory.  Ann.  Rev.  Psychol.,  1-39* 

The  author  focuses  on  the  psychological  aspects  of  individual  judgment  and 
decision  making  by  reviewing  descriptive  research  and  examining  decision  aids. 

A  survey  of  publications  (the  majority  of  them  dated  1971-75)  cn  the  subject  of 
behavioral  decision  theory  is  provided  and  descriptive  studies  of  judgment  are 
evaluated.  An  attempt  is  made  to  enumerate  the  errors  in  decision-making  that 
may  arise  from  judgment  biases.  Some  areas  mentioned  under  the  topic  of  de¬ 
scriptive  research  include  probabilistic  judgment,  choice,  models  of  risky 
choice,  and  regression  approaches.  An  insightful  discussion  of  decision  aids, 
MAUT  research,  decision  analysis,  man/machine  systems,  and  use  of  decision  aids 
clarifies:  (a)  issues  which  a  decision  maker  may  face,  and  (b)  the  systems 

used  to  solve  problems. 

Smith,  S.L.,  &  Mosier,  J.N.  (1984,  September).  Design  guidelines  for  user- 
system  interface  software  (Technical  Paper  ESD-TR-84-190).  Bedford,  Massachu- 
setts:  The  Mitre  Corporation. 

Provides  guidelines  for  the  design  of  user  interface  software  to  aid  de¬ 
signers,  system  analysts,  teachers,  students,  human  factors  practitioners,  and 
researchers  in  making  this  process  more  efficient.  The  focus  is  on  six  func¬ 
tional  areas:  data  entry,  data  display,  sequence  control,  user  guidance,  data 
transmission,  and  data  protection. 

Stevens,  A.,  Collins,  A.,  &  Goldin,  S.E.  (1979).  Misconceptions  in  student’s 
understanding.  Int.  J.  Man-Machine  Studies,  1 1 ,  145-156. 

Proposed  in  this  research  report  is  a  system  which  tutors  students  with 
intelligent  computer-aided  instruction  (iCAl)  by  diagnosing  and  rectifying 
conceptual  misunderstandings.  A  theory  based  on  examination  of  tutoring 
dialogs  is  the  basis  for  this  system.  The  goal  structure  of  a  tutor  and  types 
of  conceputal  bugs  are  specified. 

Waldrop,  G.P.,  &  Lane,  N.E.  (1985).  Computer-based  instruction  (CBl):  toward 
a  user-oriented  technology  data  base.  Proceedings  of  the  Human  Factors 
Society  -  29th  Annual  Meeting. 

The  focus  of  this  paper  is  on  two  issues  originating  from  computer-based 
instruction  (CBl)  research  and  development  analysis  in  support  of  the  CBI  data 
base  design.  The  two  points  of  concern  deal  with:  (a)  the  importance  of 
specific  attention  to  the  characteristics  and  requirements  of  potential  users 
of  CBI  and  of  the  data  base),  and  (b)  the  necessity  of  having  a  well- formulated 
classification  schema  for  description  of  the  features  of  CBI  systems  in  the 
form  of  data  storage  and  retrieval  cues.  The  goal  of  this  technology  analysis 
and  synthesis  is  to  describe  current  and  near  future  capabilities  of  CBI  in  the 
Department  of  Defense  (DOD)  environment. 
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Weisbrod,  R.L.,  Davis,  K.B.,  &  Freedy,  A.  (1975).  Adaptive  utility  assessment 
in  dynamic  decision  processes:  an  experimental  evaluation  of  decision  aiding. 
Proceedings  of  Conference  on  Systems,  Man,  and  Cybernetics  Society,  San 
Francisco,  California. 

Supports  usage  of  an  Adaptive  Dynamic  Decision  Aiding  Mechanism  (ADDAM) 
system  to  aid  decision  makers  in  their  administration  of  dynamic  decision 
tasks.  The  effectiveness  of  the  decision  aiding  system  was  examined  through  an 
experimental  study  which  used  a  realistic  decision  task.  ADDAM  allows  the 
operator  to  organize  his  own  in-context  behavior  into  a  methodical  mathematical 
framework.  The  purpose  of  this  strategy  is  not  simply  to  present  a  model  for 
decision  making  behavior,  but  also  to  furnish  a  foundation  for  decision  aiding. 


Specific  Aids 

Beach,  L.R.,  Townes,  B.D.,  Campbell,  F.L.,  &  Keating,  G.W.  (1976).  Developing 
and  testing  a  decision  aid  for  birth  planning  decisions.  Organizational 
Behavior  and  Human  Performance,  15,  99-116. 

A  decision  aid  is  promoted  which  allows  one  to  calculate  subjective  ex¬ 
pected  utilities  (SEUs)  for  having  a  (another)  child;  a  value  hierarchy  is 
used.  The  method  is  intended  to  assist  people  with  their  birth  planning  deci¬ 
sions  so  that  they  may  logically  and  sensibly  weigh  the  pros  and  cons.  Deci¬ 
sions  are  divided  into  "manageable  chunks"  and  the  relative  utility  of  each 
chunk  is  evaluated.  The  authors  affirm  that  through  the  development  of  this 
aid  they  developed  a  method  applicable  to  other  areas  besides  birth  planning. 

Brooke,  J.B.,  Duncan,  K.D.,  &  Marshall,  E.C.  (1978).  Interactive  instruction 
in  solving  fault  finding  problems.  Int.  J.  Man-Machine  Studies,  10,  605-611. 

This  study  reviews  a  training  program  which  furnishes  supplementary  infor¬ 
mation  pertaining  to  the  relationship  between  remaining  faults  and  the  real¬ 
izable  indicators  during  fault  diagnosis.  An  interactive  computer  system 
provides  problem  solving  process  information  to  trainees  as  they  learn  to  diag¬ 
nose  faults.  A  flow  chart  is  given  which  displays  the  procedure  followed  by 
computer  programmers  in  controlling  a  fault  diagnosis  problem. 

Newsted,  P.R. ,  &  Wynne,  B.E.  (1976).  Augmenting  man's  judgment  with  inter¬ 
active  computer  systems.  Int.  J.  Man-Machine  Studies,  8,  29-59. 

Presents  a  computer-based  technique,  An  Interactive  Decision  System  (AIDS), 
with  the  goal  of  assisting  typical  managers  in  making  strategic  decisions  deal¬ 
ing  with  qualitative  issues  by  allowing  them  optimal  usage  of  their  available 
information  and  resources.  This  system  is  based  on  Kepner  and  Tregoe's  infor¬ 
mation  processing  techniques  utilized  in  their  decision  training  agendas.  A 
thorough  illustration  of  the  nine  steps  in  AIDS  and  a  discussion  of  the  program 
and  its  effectiveness  describe  this  approach. 
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O'Shea,  T.  (1978).  A  self- improving  quadratic  tutor.  Int.  J.  Man-Machine 
Studies,  1 1 ,  97-124. 

The  lack  of  learning  capability  in  Computer  Assisted  Instruction  (CAI) 
programs  is  the  issue  addressed  in  this  research  report.  The  author's  goal  is 
to  present  an  alternative  teaching  strategy  for  instructing  students  on  ways  to 
solve  quadratic  equations.  Introduced  is  a  self-improving  quadratic  tutor 
which  teaches  the  solution  to  quadratic  equations  through  the  discovery  method. 
The  two  major  components  of  the  tutor  are  explained.  One  of  the  components  is 
an  adaptive  teaching  program  where  the  teaching  strategy  is  described  as  a  set 
of  production  rules,  and  the  second  component  executes  the  self- improving  func¬ 
tion  of  the  system  by  making  experimental  changes  to  the  set  of  production 
rules. 

Pangaro,  P.A.  &  Nicoll,  J.F.  (1983,  February).  Deleting  the  knowledge  engi¬ 
neer:  the  practical  design  of  intelligent  support  software  based  on  Pask's 
proto-logic  Lp. Paper  presented  at  the  Seminar  "The  Application  of  Machine  In¬ 
telligence  to  Defense  Systems",  Portland,  UK. 

Discusses  the  role  of  the  knowledge  engineer,  a  person  who  builds  an  expert 
system  in  conjunction  with  an  expert  in  the  subject  field.  Believes  that  the 
expert  should  communicate  directly  with  the  computer,  thus  eliminating  the 
need  for  a  knowledge  engineer.  An  Intelligent  Support  System  (ISS)  is  pre¬ 
sented  as  an  ideal  solution  to  providing  sophisticated  support  in  expert  sys¬ 
tems  in  three  different  roles:  the  training  of  novices,  tactical  or  opera¬ 
tional  support,  and  strategic  planning.  The  philosopy  is  that  there  should  be 
only  a  narrow  gap  between  the  creators  and  operators  of  an  intelligent  support 
system. 

Posa,  T.  (1985,  October).  Software  helps  busy  executives  analyze,  make  key 
decisions.  Infoworld,  7/41 ),  43-44. 

Decisionmap,  published  by  Softstyle  Inc.,  is  decision  support  software  (run 
on  the  Macintosh)  through  which  one  is  able  to  design  his/her  own  decision 
model.  After  choosing  criteria  on  which  the  decision  is  to  be  based,  factors 
are  weighed  according  to  importance  and  possible  solutions  are  ranked.  The 
author  advocates  its  usage,  emphasizing  the  ease  of  learning  and  minimal  amount 
of  time  involved. 

Rouse,  S.H.,  Rouse,  W.B.,  &  Hammer,  J.M.  (1982,  August).  Design  and  evaluation 
of  an  onboard  computer-based  information  system  for  aircraft.  IEEE 
Transactions  on  Systems,  Man,  and  Cybernetics,  SMC-12(4) ,  451-463. 

This  report  concerns  computer-aided  human  information  seeking.  It  is  pro¬ 
posed  that  a  computer-based  information  system  can  considerably  reduce  the 
frequency  of  human  errors.  Various  types  of  information  and  styles  of  presen¬ 
tation  are  evaluated.  Results  are  submitted  which  support  the  hypothesis  that 
computer  aiding  is  often  more  desirable  than  computerizing  in  that  the  goal 
should  be  for  computers  to  help  humans  overcome  limitations  and  utilize  their 
best  skills,  rather  than  replace  human  operators.  The  design  of  an  onboard 
computer-based  information  system  for  aircraft  is  explained. 
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Saleh,  J.f  Leal,  A.,  Kim,  J.,  &  Pearl,  J.  (1979,  October).  Progress  toward  a 
goal-directed  decision  support  sytem  (Technical  Report  UCLA-ENG-CSL-7973 ) • 
University  of  California,  Los  Angeles:  School  of  Engineering  and  Applied 
Spence. 

A  computerized  decision  support  system  is  described  which  was  developed  by 
the  authors  as  an  effective  replacement  for  decision-tree  representation.  They 
state  that  the  approach  is  preferable  because  of  the  clarity  and  purposefulness 
it  provides.  This  goal-directed  method  focuses  on  a  single  objective  at  any 
given  time  and  instructs  the  user  to  ignore  all  side  effects.  An  extensive 
amount  of  graphs,  structures,  and  representations  are  provided. 

Weisbrod,  R.L.,  Davis,  K.B.,  &  Freedy,  A.  (1975).  Adaptive  utility  assessment 
in  dynamic  decision  processes:  an  experimental  evaluation  of  decision  aiding. 
Proceedings  of  Conference  on  Systems,  Man,  and  Cybernetics  Society,  San 
Francisco,  California. 

Supports  usage  of  an  Adaptive  Dynamic  Decision  Aiding  Mechanism  (ADDAM) 
system  to  aid  decision  makers  in  their  administration  of  dynamic  decision 
tasks.  The  effectiveness  of  the  decision  aiding  system  was  examined  through  an 
experimental  study  which  used  a  realistic  decision  task.  ADDAM  allows  the 
operator  to  organize  his  own  in-context  behavior  into  a  methodical  mathematical 
framework.  The  purpose  of  this  strategy  is  not  simply  to  present  a  model  for 
decision  making  behavior,  but  also  to  furnish  a  foundation  for  decision  aiding. 


EVALUATION 


Human  Factors 


Andriole,  S.J.  (1982,  July/ August) .  The  design  of  microcomputer- based  personal 
decision  aiding  systems.  IEEE  Transactions  on  Systems,  Man,  and  Cybernetics, 
SMC-12(4),  465-469. 

Presents  an  approach  to  the  design  and  development  of  personal  computer- 
based  decision  aiding  systems  which  views  the  human  problem-solver  as  an  active 
user  who  must  be  able  to  use  the  computer  without  the  aid  of  a  problem-solving 
intermediary.  In  order  to  successfully  utilize  a  system,  the  author  states 
that  the  design  of  the  system  should  encompass  the  measurement  and  integration 
of  cognitive,  situational,  and  physiological  variables  which  should  inform  and 
alter  the  decision-aiding  process.  It  is  suggested  that  without  explicit  user 
requirements,  interactive  systems  fulfill  the  needs  of  the  designer  and  not  the 
target  user.  Problem  solvers  are  categorized  according  to  their  experience 
with  computer-based  systems  and  grouped  according  to  their  "cognitive  styles" 
in  order  to  formulate  computer-generated  information  and  strengthen  man's  in¬ 
teraction  with  the  computer.  The  user's  physiological  state  and  situational 
characteristics  are  examined  as  well. 
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Benel,  D.C.R.,  &  Pain,  R.F.  (1985).  The  human  factors  usability  laboratory  in 
product  evaluation.  Proceedings  of  the  Human  Factors  Society  -  29th  Annual 
Meeting. 

Human  factors  engineering  is  a  discipline  which  has  not  had  widespread 
application  in  commercial  product  development,  but  through  the  proposed  usa¬ 
bility  laboratory,  experts  may  incorporate  this  approach  into  their  product 
evaluation.  Describes  the  structure  of  a  usability  laboratory. 

Eklundh,  K.S.,  Marmolin,  H. ,  &  Hedin,  C.  (1985).  Experimental  evaluation  of 
dialogue  types  for  data  entry.  International  Journal  of  Man-Machine  Studies, 
22,  651-661. 

A  definite  interaction  between  type  of  dialogue  and  type  of  data  is  evident 
in  an  experiment  which  evaluated  four  fundamental  types  of  data  entry  dialogue 
concerning  speed.  It  was  found  that  subjects  in  this  experiment  preferred  an 
interaction  mode  which  maximized  speed  at  the  cost  of  less  feedback  and  less 
opportunity  for  error  control  in  the  command  dialogues.  There  were  no  signifi¬ 
cant  differences  between  the  number  of  errors  within  each  type  of  dialogue. 
Authors  contend  that  an  adaptable  man-computer  dialogue  is  necessary  with  an 
irregular  data  structure  in  performing  data  entry. 

Morehead,  D.R.  &  Rouse,  V.B.  (1985).  Human-computer  interaction  in  information 
seeking  tasks.  Information  Processing  and  Management,  19(4),  245-253. 

In  this  report,  the  authors  utilize  a  simplified  information  seeking  envi¬ 
ronment,  DBASE  (Data  Base  Access  and  Search  Environment),  in  an  experiment  to 
study  human  machine  interaction.  Included  in  this  topic  are  computer  aiding, 
availability  of  citation  lists,  data  base  structure,  and  type  of  search  tasks. 
It  is  indicated  from  the  results  of  this  experiment  that  data  base  structure 
and  type  of  search  task  may  be  combined  to  create  diversified  search  environ¬ 
ments,  which  greatly  influence  the  user's  search  performance. 


Information 

Baker,  J.D.  (1970).  Quantitative  modelling  of  human  performance  in  information 
systems.  Ergonomics,  15(6),  645-664. 

Describes  a  general  information  system  model,  MAN,  which  emphasizes  the 
importance  of  the  user  in  man-machine  systems.  The  ultimate  goal  is  to  acquire 
the  ability  to  utilize  the  results  of  human  information  processing  studies  to 
predict  the  impact  on  system  performance.  The  framework  structure  is  based  on 
the  supposition  that  there  are  five  central  operations  in  the  performance  of  an 
information  system:  screen,  transform,  input,  assimilate,  and  decide. 

Bandyopadhyay,  R.  (1977,  January).  Information  for  organizational  decision¬ 
making  -  a  literature  review.  IEEE  Transactions  on  Systems,  Man,  and 
Cybernetics,  SMC-7 (1 ) ,  1-15. 

Classifies  literature  pertaining  to  organizational  decision  making  into 
three  areas:  management  science,  classical,  and  behavioral  science.  Stresses 
that  there  is  a  problem  in  measuring  information  value.  The  author  aims  to 
provide  insight  into  the  broad  area  of  organizational  decisionmaking. 
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Benbasat,  I.  &  Taylor,  R.N.  (1982,  July/ August) .  Behavioral  aspects  of  infor¬ 
mation  processing  for  the  design  of  management  information  systems.  IEEE 
Transactions  on  Systems,  Man,  and  Cybernetics,  SMC-1 2(4) ,  439-450. 

Cites  various  models  of  information  processing  which  account  for  human 
thinking  and  attempts  to  cover  the  capabilities,  as  well  as  the  inadequacies, 
in  information  processing  present  in  the  literature.  The  authors  believe  that 
management  information  systems  (MIS)  can  be  radically  improved  by  understanding 
the  behavioral  processes  by  which  humans  process  information  and  make  choices. 
This  will  aid  in  eliminating  the  problem  of  "overloading"  the  human  user. 

Cohen,  M.S.,  &  Freeling,  A.N.S.  (1981,  February).  The  impact  of  information  on 
decisions:  command  and  control  system  evaluation  (Technical  Report  81 —1 ) . 

Falls  Church,  Virginia:  Decision  Science  Consortium,  Inc. 

This  study  applies  decision  analytic  concepts  to  the  problem  of  determining 
value  of  information  in  the  design  and  evaluation  of  information  systems. 
Although  many  current  systems  furnish  an  abundance  of  data,  the  decision  maker 
is  often  unable  to  pick  out  the  data  essential  to  the  problem.  Neoeusary  in¬ 
formation  is  that  which  has  a  high  probability  of  changing  a  decision  and  im¬ 
proving  decision  outcome.  This  study  modifies  standard  VOI  techniques  in  order 
to  simplify  them  and  encourage  their  usage. 

Connolly,  T. ,  &  Miklausich,  V.M.  (1978).  Some  effects  of  feedback  error  in 
diagnostic  decision  tasks.  Academy  of  Management  Journal,  21 (2) ,  301-307. 

A  laboratory  study  was  conducted  to  examine  the  effects  of  varying  the 
levels  of  feedback  error  on  performance  in  diagnostic  tasks.  Results  suggest 
that  the  comprehensive  decision  quality  is  contingent  on  both  the  quality  of 
information  available  when  the  decision  is  made  and  the  quality  of  subsequent 
feedback. 

Klayman,  J.  (1982).  Simulations  of  six  decision  strategies:  comparisons  of 
search  patterns,  processing  characteristics,  and  response  to  task  complexity. 
(Working  paper  #100). Chicago,  Illinois: University  of  Chicago. 

This  paper  addresses  questions  pertaining  to  the  modifications  made  in 
strategies  in  response  to  task  complexity  by  decision  makers.  A  primary  goal 
of  these  simulation  studies  entails  identifying  different  decision  strategies 
on  the  basis  of  information  search  patterns.  It  is  reasoned  that  simulation 
can  be  very  useful  in  generating,  testing,  and  modifying  hypotheses  about  human 
decision  strategies  but  care  must  be  taken  to  insure  that  the  design  is  repre¬ 
sentative  of  the  impending  specific  decision  task. 

Klayman,  J.  (1983).  Analysis  of  predecisional  information  search  patterns. 

In  P.  Humphreys,  0.  Svenson,  &  A.  Vari  (Eds.),  Analyzing  and  Aiding  Decision 
Processes.  Amsterdam:  North-Holland. 

The  present  paper  discusses  the  analysis  of  information  gathering  patterns 
as  a  tool  for  process  tracing.  Attention  to  complex  and  diverse  decision 
approaches  has  underscored  the  necessity  of  developing  more  refined  process 
tracing  analyses  (analyzing  information  gathering  patterns).  Two  major  methods 
of  improving  search  analysis  are  examined.  An  experimental  example  using  these 
techniques  is  given,  and  applications  are  submitted  in  three  areas. 
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Klein,  G.A.  (1980,  October).  Automated  aids  for  the  proficient  decision  maker. 
Proceedings  of  the  Smith  Conference  on  Cybernetics  and  Society. 

Proposes  a  comparison- guided  model  of  decision-making,  maintaining  that 
proficient  decision  makers  are  able  to  recognize  and  subsequently  place  new 
situations  parallel  to  ones  previously  experienced.  The  types  of  decisions 
discussed  are  complex,  those  with  unusual  conditions.  It  is  asserted  that  a 
redefinition  of  the  problem  by  the  expert  will  allow  utilization  of  all  re¬ 
sources  which  may  yield  a  solution  to  the  problem,  rather  than  using  a  decision 
aid  requiring  "rule-following."  The  author  believes  that  proficiency  is  not 
simply  following  rules  and  procedures. 

Lichtenstein,  S. ,  &  Slovic,  P.  (1973).  Response-induced  reversals  of  pref¬ 
erence  in  gambling:  an  extended  replication  in  Las  Vegas.  Journal  of  Experi¬ 
mental  Psychology,  101 ( 1 ) ,  16-20. 

An  experiment  was  conducted  at  the  Four  Queens  Casino  in  Las  Vegas  to  de¬ 
termine  the  effects  of  response  mode  upon  information-processing  strategies 
applied  by  subjects  in  gambling  decisions.  Different  strategies  were  used  for 
choosing  among  pairs  of  bets  as  opposed  to  assigning  monetary  values  to  single 
bets.  Bias  was  found  to  occur  in  favorable  and  unfavorable  gambles  as  a  result 
of  cue-response  compatibility. 

Morehead,  D.R.,  &  Rouse,  W.B.  (1983).  Human- computer  interaction  in  informa¬ 
tion  seeking  tasks.  Information  Processing  and  Management,  19(4),  243-253. 

In  this  report,  the  authors  utilize  a  simplified  information  seeking  envi¬ 
ronment,  DBASE  (Data  Base  Access  and  Search  Environment),  in  an  experiment  to 
study  human  machine  interaction.  Included  in  this  topic  are  computer  aiding, 
availability  of  citation  lists,  data  base  structure,  and  type  of  search  tasks. 
It  is  indicated  from  the  results  of  this  experiment  that  data  base  structure 
and  type  of  search  task  may  be  combined  to  create  diversified  search  environ¬ 
ments,  which  greatly  influence  the  user's  search  performance. 

Newsted,  P.R.,  &  Wynne,  B.E.  (1976).  Augmenting  man's  judgment  with  inter¬ 
active  computer  systems.  Int.  J.  Man-Machine  Studies,  £3,  29-59. 

Presents  a  computer-based  technique,  An  Interactive  Decision  System  (AIDS), 
with  the  goal  of  assisting  typical  managers  in  making  strategic  decisions  deal¬ 
ing  with  qualitative  issues  by  allowing  them  optimal  usage  of  their  available 
information  and  resources.  This  system  is  based  on  Kepner  and  Tregoe's  infor¬ 
mation  processing  techniques  utilized  in  their  decision  training  agendas.  A 
thorough  illustration  of  the  nine  steps  in  AIDS  and  a  discussion  of  the  program 
and  its  effectiveness  describe  this  approach. 

Norman,  D.A.  (1980,  August).  Errors  in  human  performance  (Report  No.  8004). 

La  Jolla,  California:  Center  for  Human  Information  Processing. 

This  report's  area  of  concentration  lies  in  human  error,  particularly  error 
made  by  skilled  operators  in  complex,  demanding  systems.  Through  this  analysis 
of  human  error,  various  applications  of  human  information  processing  are  de¬ 
lineated  and  a  classification  scheme  for  errors  is  outlined.  Short-term  memory 
and  human  attentional  limitations  are  examined  in  the  design  of  human-mach  .ne 
interfaces. 
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Norman,  D.A.  (1981b).  A  psychologist  views  human  processing;  human  errors  and 
other  phenomena  suggest  processing  mechanisms.  Paper  presented  at:  IJCAI, 
Vancouver,  British  Columbia. 

Discusses  human  processing  structures  by  arguing  for  a  different  kind  of 
processing  mechanism  than  that  which  is  most  commonly  referred  to  in  Artifi¬ 
cial  Intelligence.  The  form  of  human  error  is  of  major  importance  in  this 
study.  A  distinction  is  made  between  two  major  classes  of  errors,  mistakes 
versus  slips,  and  these  two  types  are  examined. 

Pitz,  G.F.  (1970).  On  the  processing  of  information:  probabilistic  and  other¬ 
wise.  In  G.  de  Zeeuw  et  al.  (Eds.),  Acta  Psychologica  34  Subjective  Probabil¬ 
ity  (pp.  201-213)*  Amsterdam:  North-Holland  Publishing  Company. 

Advocated  is  the  study  of  processes  such  as  attention,  encoding,  and  short¬ 
term  memory  in  relation  to  probabilistic  information  processing  (PIP)  in  order 
to  expand  our  knowledge  of  decision  making  behavior.  It  is  stressed  that  the 
information,  not  the  processing  itself,  is  probabilistic  in  nature.  Two  models 
of  opinion  revision  are  reviewed  which  incorporate  the  use  of  Bayes'  theorem  in 
their  models,  but  recognize  human  error.  Outlined  is  a  PIP  model  computer 
program. 

Rouse,  S.H.,  Rouse,  W.B.,  &  Hammer,  J.M.  (1982,  August).  Design  and  evaluation 
of  an  onboard  computer-based  information  system  for  aircraft.  IEEE 
Transactions  on  Systems,  Man,  and  Cybernetics,  SMC-12(4),  451 -463. 

This  report  concerns  computer-aided  human  information  seeking.  It  is  pro¬ 
posed  that  a  computer-based  information  system  can  considerably  reduce  the 
frequency  of  human  errors.  Various  types  of  information  and  styles  of  presen¬ 
tation  are  evaluated.  Results  are  submitted  which  support  the  hypothesis  that 
computer  aiding  is  often  more  desirable  than  computerizing  in  that  the  goal 
should  be  for  computers  to  help  humans  overcome  limitations  and  utilize  their 
best  skills,  rather  than  replace  human  operators.  The  design  of  an  onboard 
computer-based  information  system  for  aircraft  is  explained. 

Rouse,  W.B.,  &  Rouse,  S.H.  (1984).  Human  information  seeking  and  design  of 
information  systems.  Information  Processing  and  Management,  20 (l ) ,  129-158. 

Organizes  current  literature  from  various  disciplines  (i.e.,  psychology, 
management,  systems  engineering,  computer  science,  and  library  science)  to 
develop  the  topic  of  human  information  seeking  and  information  systems.  The 
nature  of  information  seeking  is  defined,  and  through  discussion,  the  contrast 
is  made  between  information  seeking  and  information  processing.  The  difficulty 
in  defining  and  measuring  the  value  of  information  affects  the  progress  which 
may  be  made  in  developing  frameworks  and  formulating  methods  to  evaluate  this 
area  of  concern.  Human  information  seeking  is  shown  tc  be  a  integral  part  of 
decision  making  and  problem  solving. 
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Schweiger,  D.M.,  Anderson,  C.R.,  &  Locke,  E.A.  (1985)*  Complex  decision  mak¬ 
ing:  a  longitudinal  study  of  process  and  performance.  Organizational  Behav¬ 
ior  and  Human  Decision  Processes,  36,  245-272. 

Simultaneous  verbal  protocols  were  employed  to  study  individual  decision 
making  processes.  The  authors  describe  the  protocol  analysis  procedure  used 
and  present  the  coding  scheme  developed  to  analyze  the  data.  Results  of  two 
experiments  show  that  thought  processes  used  in  complex  decision  making  are 
associated  with  subsequent  performance:  subjects  using  causal  analysis  per¬ 
formed  more  successfully  than  those  who  did  not. 

Shanteau,  J.  (1974).  Component  processes  in  risky  decision  making.  Journal  of 
Experimental  Psychology,  103(4),  680-691. 

The  goal  of  this  research  study  was  to  utilize  the  theory  of  information 
integration  by  applying  an  information  processing  approach  to  risky  decision 
judgments.  The  study  challenges  the  utility  theory  as  used  in  risky  decision 
making.  The  multiplying  model  is  advocated,  which  is  supported  by  data 
resulting  from  two  experiments. 

Shanteau,  J.  (1975).  Averaging  versus  multiplying  combination  rules  of  infer¬ 
ence  judgment.  Acta  Psychologies,  39,  83-89. 

Two  combination  rules,  averaging  and  multiplying,  which  describe  sequential 
inference  judgments,  were  compared  through  three  experiments  in  this  study. 

Both  of  these  rules  depict  how  successive  pieces  of  evidence  are  combined  with 
previous  judgments  to  form  new  judgments.  Usage  of  the  averaging  rule  was  sup¬ 
ported  but  the  descriptive  ability  of  the  Bayesian  approach  (and  other 
approaches  which  use  the  multiplying  rule)  was  questioned.  It  may  be  inferred 
from  the  averaging  result  that  judgments  may  incorporate  processing  rules  dif¬ 
ferently  than  previously  speculated. 

Shanteau,  J. ,  &  Gaeth,  G.J.  (1983,  January).  Training  expert  decision  makers 
to  ignore  irrelevant  information  (Human  Information  Processing  Report  No. 

82-5) •  Manhattan,  Kansas:  Kansas  State  University. 

A  series  of  research  projects  were  performed  with  the  goal  of  revealing  the 
influence  of  irrelevant  information  on  expert  decision  makers.  Training 
programs  to  overcome  this  effect  were  discussed  and  the  long-term  effects  of 
the  training  were  evaluated. 

Shanteau,  J. ,  &  Nagy,  G.F.  (1984).  Information  integration  in  person  percep¬ 
tion:  theory  and  application.  In  M.  Cook  (Ed.),  Issues  in  person  perception 
(pp.  48-86).  London:  Methuen. 

This  selected  chapter  covers  some  of  the  principal  findings  from  research 
conducted  on  the  person  perception  model  with  the  information  integration  the¬ 
ory  (ITT).  Five  research  studies  are  alluded  to  which  exemplify  the  early  de¬ 
velopment  and  basic  assumptions  of  ITT,  which  is  traced  to  impression 
formation.  Cases  are  cited  which  show  how  ITT  can  be  utilized  in  the  analysis 
of  socially  relevant  interpersonal  decisions,  and  also  in  real-world  concerns. 
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Slovic,  P.  (1971,  April).  Information  processing,  situation  specificity,  and 
the  generality  of  risk-taking  behavior.  ORI  Research  Bulletin,  11(3),  1-22. 

Inter-task  consistency  of  individual  differences  was  ascertained  in  two 
structurally  comparable  risk-taking  tasks.  A  relationship  was  shown  between 
information  processing  and  situational  specificity.  As  a  result  of  the  experi¬ 
ment,  it  was  determined  that  the  two  response  modes  triggered  different  methods 
of  processing  information  pertaining  to  probabilities  and  payoffs  in  a  way  that 
influenced  individual  differences  and  reduced  inter-task  consistencies.  The 
implication  is  that  it  is  improbable  to  find  significant  correlations  between 
risk-taking  measures  in  structurally  different  settings  or  between  risk-taking 
and  different  behaviors. 

Slovic,  P. ,  &  Lichtenstein,  S.  (1968,  November).  Relative  importance  of 
probabilities  and  payoffs  in  risk  taking.  Journal  of  Experimental  Psychology 
Monograph,  78(3),  1-18. 

This  report  examines  the  beliefs  held  by  people  about  the  relevant  impor¬ 
tance  of  probabilities  and  payoffs  and  their  capability  to  act  on  the  premise 
of  these  beliefs  when  processing  information  included  in  the  description  of  a 
gamble.  Two  experiments  are  described  which  suggest  that  there  is  utility  in 
recognizing  decisions  about  gambling  within  the  context  of  information  process¬ 
ing. 


General 


Adelman,  L. ,  Donnell,  M.L.,  Patterson,  J.F.,  &  Weiss,  J.J.  (1981,  February). 
Issues  in  the  design  and  evaluation  of  decision-analytic  aids  (Technical 
Report  81-1-304). McLean,  Virginia:  Decisions  and  Designs,  Inc. 

Addresses  three  important  issues  which  are  relevant  to  the  design  and 
evaluation  of  decision  analytic  aids  and  presents  a  framework  which  illustrates 
the  decision  making  organization.  The  three  interfaces  identified  are:  be¬ 
tween  the  decision  aid  and  the  user,  between  the  user  and  the  extent  to  which 
the  decision  aid  facilitates  the  decision  making  processes  of  the  organization, 
and  between  the  decision  making  organization  and  whether  or  not  the  quality  of 
the  organization's  decision  making  has  improved  the  environment.  Without  the 
smooth  functioning  of  the  three  interfaces,  effective  integration  of  decision 
aids  into  organizations  can  not  be  accomplished.  The  author  covers  some  ways 
to  attract  the  user  to  the  aid  and  to  simplify  its  overall  operation. 

Beach,  L.R.,  Townes,  B.D.,  Campbell,  F.L.,  &  Keating,  G.W.  (1976).  Developing 
and  testing  a  decision  aid  for  birth  planning  decisions.  Organizational 
Behavior  and  Human  Performance,  13,  99-116. 

A  decision  aid  is  promoted  which  allows  one  to  calculate  subjective  ex¬ 
pected  utilities  (SEUs)  for  having  a  (another)  child;  a  value  hierarchy  is 
used.  The  method  is  intended  to  assist  people  with  their  birth  planning  deci¬ 
sions  so  that  they  may  logically  and  sensibly  weigh  the  pros  and  cons.  Deci¬ 
sions  are  divided  into  "manageable  chunks"  and  the  relative  utility  of  each 
chunk  is  evaluated.  The  authors  affirm  that  through  the  development  of  this 
aid  they  developed  a  method  applicable  to  other  areas  besides  birth  planning. 
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Charlton,  S.G.  (1985).  Behavior  analysis:  a  tool  for  test  and  evaluation. 
Proceedings  of  the  Human  Factors  Society  -  29th  Annual  Meeting. 

In  this  study  behavioral  analysis  is  applied  in  the  field  test  environment. 
Used  simultaneously  with  other  evaluation  tools,  behavior  changes  can  be  fully 
evaluated.  In  the  example  provided,  behavior  analysis  was  used  to  document  and 
assess  instances  of  behavioral  divergence  in  the  field  test  of  a  militai’y  sys¬ 
tem. 

Cohen,  M.S.,  £  Freeling,  A.N.S.  (1981,  February).  The  impact  of  information  on 
decisions:  command  and  control  system  evaluation  (Technical  Report  81-1). 

Falls  Church,  Virginia:  Decision  Science  Consortium,  Inc. 

This  study  applies  decision  analytic  concepts  to  the  problem  of  determining 
value  of  information  in  the  design  and  evaluation  of  information  systems. 
Although  many  current  systems  furnish  an  abundance  of  data,  the  decision  maker 
is  often  unable  to  pick  out  the  data  essential  to  the  problem.  Necessary  in¬ 
formation  is  that  which  has  a  high  probability  of  changing  a  decision  and  im¬ 
proving  decision  outcome.  This  study  modifies  standard  VOI  techniques  in  order 
to  simplify  them  and  encourage  their  usage. 

Dino,  G.A. ,  &  Shanteau,  J.  (1984,  November).  What  skills  do  managers  consider 
important  for  effective  decision  making?  Paper  presented  at  the  Psychonomics 
Meeting,  San  Antonio,  Texas. 

This  study  was  conducted  with  the  purpose  of  analyzing  how  decision  makers 
themselves  define  decision  making  competence.  Thirty- five  bottom-level  and 
twenty-four  top-level  managers  participated  in  the  following:  (l)  Rated  skills 
that  they  believed  important  for  effective  decision  makers;  (2)  Rated  the 
importance  of  these  skills  in  upper-level  and  lower-level  decision  makers;  and, 
(3)  Were  evaluated  in  regards  to  the  effects  of  gender,  their  types  of 
organizations,  and  their  levels  within  their  organizations. 

Einhorn,  H.J.,  Hogarth,  R.M. ,  &  Klempner,  E.  (1977).  Quality  of  group  judg¬ 
ment.  Psychological  Bulletin,  84 ,  158-172. 

Deals  with  the  accuracy  of  group  judgment  within  and  between  different 
populations  of  groups.  The  quality  of  group  judgment  is  defined.  Results  as 
well  as  constraints  affecting  judgmental  quality  of  various  strategies  for 
merging  opinions  under  a  diversity  of  circumstances  are  discussed.  A  statisti¬ 
cal  procedure  is  introduced  that  considers  which  baselines  are  suitable  in  the 
evaluation  of  the  quality  of  group  judgment. 

Eklundh,  K.S.,  Marmolin,  H. ,  &  Hedin,  C.  (1985).  Experimental  evaluation  of 
dialogue  types  for  data  entry.  International  Journal  of  Man-Machine  Studies, 
22,  651-661. 

A  definite  interaction  between  type  of  dialogue  and  type  of  data  is  evident 
in  an  experiment  which  evaluated  four  fundamental  types  of  data  entry  dialogue 
concerning  speed.  It  was  found  that  subjects  in  this  experiment  preferred  an 
interaction  mode  which  maximized  speed  at  the  cost  of  less  feedback  and  less 
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opportunity  for  error  control  in  the  command  dialogues.  There  were  no  signifi¬ 
cant  differences  between  the  number  of  errors  within  each  type  of  dialogue. 
Authors  contend  that  an  adaptable  man-computer  dialogue  is  necessary  with  an 
irregular  data  structure  in  performing  data  entry. 

Fischhoff,  B.  (1977,  May).  Decision  analysis:  clinical  art  or  clinical  sci¬ 
ence?  (Technical  Report  PTR-1 042-77-5 )  •  Eugene,  Oregon:  Decision  Research,  A 
Branch  of  Perceptronics. 

Draws  an  analogy  between  decision  analysis  and  psychotherapy.  Areas  dis¬ 
cussed  include  evaluating  the  validity,  effectiveness,  and  underlying  assump¬ 
tions  of  decision  analysis,  improving  analysts'  skills,  and  social,  ethical, 
and  political  issues.  The  comparison  with  psychotherapy  serves  to  delineate 
the  important  issues  that  should  be  addressed  in  a  research  program. 

Hart,  S.L.  (1985).  Toward  quality  criteria  for  collective  judgments. 
Organizational  Behavior  and  Human  Decision  Processes,  56,  209-228. 

Addresses  the  subject  of  group  decision  making  by  presenting  evaluative 
criteria,  which  were  derived  from  a  review  of  prevailing  literature,  for  judg¬ 
ing  when  a  "high-quality  outcome"  has  been  reached.  A  major  goal  which  is 
hoped  to  be  generated  by  this  project  is  the  construction  of  a  set  of  empiri¬ 
cally  derived  criteria  against  which  outcome  quality  may  be  judged.  The  most 
effective  procedures  for  the  achievement  of  a  high  quality  collective  judgment 
are  suggested  as  a  result  of  six  factual  group  problem-solving  cases  which 
were  evaluated  by  use  of  a  specially  developed  questionnaire. 

Hogarth,  R.M.  (1975).  Decision  time  as  a  function  of  task  complexity.  In 
Wendt/Velk  (Eds.).  Utility  Probability,  and  Human  Decision  Making  (pp.  521- 
558).  Dcrdrecht-Holland:  D.  Riedel  Publishing  Company. 

Discusses  the  association  between  decision  time  and  task  complexity.  A 
mathematical  model  is  thoroughly  explained  which  examines  this  association. 

The  model  is  described  as  having  several  observable  variables  linked  to  an 
unobservable  variable  of  the  psychological  function,  i.e.,  cognitive  strain. 

It  is  suggested  that  when  a  decision  maker  is  faced  with  different  altern¬ 
atives,  he  considers  two  kinds  of  cost:  the  time  involved  in  the  decision  pro¬ 
cedure,  and  the  cost  of  making  errors. 

Hogarth,  R.M.,  &  Makridakis,  S.  (1981 ).  The  value  of  decision  making  in  a 
complex  environment:  an  experimental  approach.  Management  Science,  27, 

95-107.  . 

An  experiment  was  conducted  in  which  a  competitive  game  was  played  to 
assess  whether  or  not  the  costs  of  time  and  effort  used  to  analyze  a  decision 
outweighed  the  benefits.  Another  objective  was  to  discover  strategies  to 
improve  decision  performance.  There  were  two  kinds  of  uncomplicated  decision 
rules:  (l  )  rules  were  applied  consistently  (arbitrary-consistent),  and  (2) 
rules  were  subject  to  a  random  component  (arbitrary- random). 
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Howard,  R.A.  (1979,  October).  An  assessment  of  decision  analysis.  Stanford, 
California:  Stanford  University. 

Presents  an  assessment  of  decision  analysis  by  discussing  human  decision 
making,  decision  analysis,  the  usefulness  of  decision  analysis,  ethical  con¬ 
cerns,  and  the  challenge  that  must  be  faced  if  the  field  is  to  continue  its 
growth.  The  author  explains  that  "decision  making  is  what  you  do  when  you  don't 
know  what  to  do",  and  cites  evidence  that  people  are  not  good  natural  decision 
makers.  It  is  emphasized  that  decision  analysis  is  not  simply  a  logical  proce¬ 
dure,  but  also  an  "artistic"  process. 

Jacobs,  T.O.  (1976,  September).  Analysis  and  evaluation  of  tactical  decision 
making.  Paper  presented  at  American  Psychological  Association  Meeting, 
Washington,  D.C. 

Outlines  a  study  in  which  an  illustration  of  how  incumbent  generals,  colo¬ 
nels,  and  lieutenant  colonels  define  command  and  control  performances.  Five 
general  categories  which  emerged  were:  getting  and  using  information,  plan¬ 
ning,  organizing,  controlling/directing,  and  leadership/personnel.  Reports  on 
an  experiment  in  which  the  usefulness  of  automating  analysis  and  production 
tasks  is  analyzed  and  proves  to  be  applicable  to  organizational  decision  mak¬ 
ing.  Among  the  findings  reported  are:  (a)  The  battalion  level  differs  from 
division  command  level  in  that  the  descriptive  model  suggests  a  "process-ori¬ 
ented  reactive  orientation"  for  battalion  level  decision  makers;  (b)  the  time 
perspective  is  very  limited  for  the  battalion  commander  in  combat;  and,  (c)  ex¬ 
periments  in  the  area  of  tactical  decision  making  may  require  the  utilization 
of  real  incumbents  in  realistic  situations. 

Keen,  P.G.W.  (1975,  Spring).  Computer-based  decision  aids:  the  evaluation 
problem.  Sloan  Management  Review,  _1_6_(3 ) »  17-29- 

Examines  the  issue  of  the  evaluation  problem  with  computer-based  decision 
aids.  The  author  proposes  that  a  diagnostic  approach  be  implemented  which  con¬ 
siders  all  benefits,  "soft"  and  "hard,"  since  it  is  not  always  feasible  that 
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60 


decision  trees;  (c)  the  application  of  value  assessment  procedures;  and,  (d)  a 
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management,  systems  engineering,  computer  science,  and  library  science)  to 
develop  the  topic  of  human  information  seeking  and  information  systems.  The 
nature  of  information  seeking  is  defined,  and  through  discussion,  the  contrast 
is  made  between  information  seeking  and  information  processing.  The  difficulty 
in  defining  and  measuring  the  value  of  information  affects  the  progress  which 
may  be  made  in  developing  frameworks  and  formulating  methods  to  evaluate  this 
area  of  concern.  Human  information  seeking  is  shown  to  be  a  integral  part  of 
decision  making  and  problem  solving. 
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Reviews  current  measurement  theories  on  utility  modeling  and  assessment. 
Written  particularly  for  decision  analysts  who  are  interested  in  these  theories 
for  deciphering  and  solving  real  world  decision  problems  which  are  complex  in 
nature.  This  paper  aims  to  relate  the  present  utility  measurement  theory 
with  decision  analytic  practice.  The  paper  tries  to  lessen  the  gap  between  the 
theory  and  practice  of  utility  measurement,  which  may  be  attributed  in  part  to 
the  difficulty  in  comprehension  of  the  mathematical  language  inherent  in  the 
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tasks.  The  effectiveness  of  the  decision  aiding  system  was  examined  through  an 
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cognitive  representation,  and  conflict.  Assuming  a  broad  perspective  in  the 
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Provides  a  summary  of  literature  which  may  be  useful  in  developing  and  im¬ 
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